DETAILED ACTION
*Note in the following document:
1. Texts in italic bold format are limitations quoted either directly or conceptually from claims/descriptions disclosed in the instant application.
2. Texts in regular italic format are quoted directly from cited reference or Applicant’s arguments.
3. Texts with underlining are added by the Examiner for emphasis.
4. Texts with
5. Acronym “PHOSITA” stands for “Person Having Ordinary Skill In The Art”.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This is in response to applicant’s amendment/response file on 6 July 2026, which has been entered and made of record. Claims 1, 7 and 15 has/have been amended. Claims 21-25 has/have been added. Claims 8-14 have been cancelled. Claims 1-7 and 16-25 are pending in the application.
Response to Arguments
Applicant's arguments, with respect to Double Patenting rejection, see p.9, filed on 6 July 2026 have been fully considered and are persuasive. The previous Double Patenting rejection is withdrawn after Terminal Disclaimer has been filed and approved.
Applicant's arguments, with respect to 35 U.S.C. §103 rejection to independent Claim 1/15, see p.10-12, filed on 6 July 2026 have been fully considered and are persuasive. The previous 35 U.S.C. §103 rejection to Claim 1/15 and their dependent claims are withdrawn after Claim 1/15 being amended.
Applicant’s arguments, see p.12-14, filed on 6 July 2026, with respect to 35 U.S.C. §103 rejection to Claim 7 have been fully considered but they are not persuasive.
Regarding Claim 7 and reference of Chen et al. (US 2017/0352196 A1), Applicant argues, on p.12:
PNG
media_image1.png
604
614
media_image1.png
Greyscale
The Examiner respectfully disagrees.
Gebbie discloses presenting the first annotation menu to the user via the transmissive display ([0017]). A skilled person would have known that the virtual annotation menu is for the user to make a selection. Therefore the virtual menu should not be blocked by any real or virtual object. A virtual annotation is a virtual object too. Since Chen teaches offsetting blocked annotation from another virtual annotation which is a virtual object as well, it would have been obvious to a PHOSITA to adopt the teaching of Chen and to add the limitation of determining whether a view of the first annotation menu is occluded by the virtual object; and in response to determining that the view of the first annotation menu is occluded by the virtual object, repositing the first annotation menu such that the first annotation menu is not occlude by the virtual object. The motivation is to make the virtual annotation menu viewable to a user for the user to make selection.
Therefore the Examiner maintains the 35 U.S.C. §103 rejection to Claim 7.
Claim Rejections - 35 USC § 103
Claims 7 is rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto et al. (US 2019/0019348 A1) in view of Gebbie et al. (US 2019/0180506 A1), Wheeler et al. (US 2013/0106674 A1) and further in view of Chen et al. (US 2017/0352196 A1).
Regarding Claim 7, Yamamoto teaches or suggests a system comprising:
a wearable device comprising a transmissive display ([0008]: FIG. 1 is an example implementation of an augmented and/or virtual reality system including a head mounted display in communication with external input devices. [0003]: An augmented reality (AR) and/or virtual reality (VR) system may generate an immersive, three-dimensional (3D) virtual environment. A user may interact with objects in this virtual environment using various electronic devices, such as, for example, a helmet or other head mounted device including a display, glasses or goggles that a user looks through when viewing a display device, one or more handheld electronic devices such as controllers, joysticks and the like, gloves fitted with sensors, keyboards, mouse, and other electronic devices); and
one or more processors (Fig.3: Processor 390 and 309) configured to execute a method comprising:
presenting a virtual object (Fig.7B: the two virtual houses shown on virtual surface 600C) to a user (Fig.7B user 1) at a first position via the transmissive display of the wearable device (Fig.7B and [0034]: [0034] As shown in FIG. 7B, the first user A (or the second user B) may select one of the virtual surfaces 600, for example, the virtual surface 600C, for review by, for example, selecting the virtual surface 600C using one, or both, of the controllers A1, A2. In reviewing the selected virtual surface 600C, the first user A may, for example, enlarge the selected virtual surface 600C, for example, in the manner described above with respect to FIGS. 5D-5E and 6A-6B. The first user A may also add notes, sketches or other annotations to the virtual surface 600C, for example, in the manner described above with respect to FIGS. 5A-5C);
PNG
media_image2.png
425
570
media_image2.png
Greyscale
in response to a first input provided at a first time by a second user (Fig.7B user B), presenting a virtual annotation at a first displacement from the first position via the transmissive display ([0035]: The second user B may view the virtual surface 600C, which has been annotated by the first user A, either substantially at the same time, in the shared virtual environment 550, or at a later time, and may further revise or otherwise annotate the virtual surface 600C);
transmitting first data to a remote server, the first data associated with the virtual object, the virtual annotation and the first displacement (Fig.9 and [0068]: the systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), and the Internet. It would have been obvious to a PHOSITA that the HMD is a client and it is the HMD user who provides the first data associated with the virtual object, the virtual annotation and the first displacement);
in response to a second input provided at a second time by the second user, presenting the virtual annotation at a second displacement from the first position via the transmissive display, the second displacement different from the first displacement ([0042]: Virtual objects may be moved, for example, from the relatively near field to the relatively far field, in a similar manner. For example, after annotation of the virtual surface 600, the user may wish to move the annotated virtual surface 600 to clear space in the near virtual field for a new task. The user may select the annotated virtual surface 600, the virtual surface 600 representing the selected virtual object in the virtual environment, as shown in FIG. 7J, for example, in the manner described above. Upon detection that the virtual surface 600 has been selected, the user may implement a gesture indicating a desired movement of the selected virtual surface 600. For example, the user implement a rotating or twisting gesture, as shown in FIG. 7K, to move the selected virtual surface 600 away from the user, from the near virtual field toward the far virtual field); and
transmitting second data to the remote server, the second data associated with the virtual object, the virtual annotation and the second displacement ([0052]: Also, multiple computing devices 1000 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system). Also see [0056], [0068]-[0069] and [0077]. It would have been obvious to a PHOSITA that the HMD is a client and it is the HMD user who provides the second data associated with the virtual object, the virtual annotation, and the second displacement);
Yamamoto fails to disclose wherein: the first input is provided via a first annotation menu dynamically presented to the second user at the first time at a fixed orientation relative to an orientation of the second user at the first time, and the second input is provided via a second annotation menu dynamically presented to the second user at the second time at a fixed orientation relative to an orientation of the second user at the second time.
However Gebbie discloses making annotation selection through virtual menu ([0037]: In some embodiments, when a user wants to add an annotation to a virtual object, the user directs the tool (e.g., handheld controller, finger, eye gaze, or similar means) to intersect with the virtual object, the intersection is detected, and the user is provided a menu to draw or attach. The menu options may be programmable into a controller, or provided a virtual menu that appears when the intersection occurs).
Wheeler discloses determining a gaze axis within a field of view of a head-mounted display (HMD), wherein the HMD is configured to display virtual images within the field of view. The method further includes determining a reference axis related to the HMD, adjusting a tracking rate based on the gaze axis and the reference axis, and moving the virtual images within the field of view based on the gaze axis, the reference axis and the tracking rate ([0004]). Wheeler discloses based on detected eye gaze, rotating virtual menu and keeping the menu at a fixed orientation relative to an orientation of the user’s eye gaze direction ([0098]: FIG. 7D illustrates an embodiment 732 where the HMD user may be travelling on the subway car at position 708. At that position 708, the HMD user is traveling forward and also rotating gradually to the left. If the user interface is not corrected by an eye-tracking system, the controller may interpret the HMD movement as an intentional movement to turn left. In general, this motion may lead to rotating the menus to the right. In this case, the photos 720 menu may be moved towards the center of the HMD display 716. However, FIG. 7E illustrates an embodiment 736 that may use an eye-tracking system to correctly determine the actual desired speed of user interface movement. In this case, the HMD user is moving through position 708, while moving forward and rotating to the left. At the same time, an eye gaze point 738 may be determined to remain at the center of the HMD display 716. The eye gaze fixation may cause the menu to not rotate).
PNG
media_image3.png
504
426
media_image3.png
Greyscale
Both Gebbie’s and Wheeler’s applications are related to using head-mounted glass in mixed reality (Gebbie Fig.1A. Wheeler Fig.2). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Gebbie and Wheeler into that of Yamamoto and to include the limitation of the first input is provided via a first annotation menu dynamically presented to the second user at the first time at a fixed orientation relative to an orientation of the second user at the first time, and the second input is provided via a second annotation menu dynamically presented to the second user at the second time at a fixed orientation relative to an orientation of the second user at the second time in order to allow a user to select a personized annotation even when the user is moving status.
Yamamoto modified by Gebbie and further teaches or suggests receiving an indication to present the first annotation menu to the user via the transmissive display (Gebbie [0037]: In some embodiments, when a user wants to add an annotation to a virtual object, the user directs the tool (e.g., handheld controller, finger, eye gaze, or similar means) to intersect with the virtual object, the intersection is detected, and the user is provided a menu to draw or attach. The menu options may be programmable into a controller, or provided a virtual menu that appears when the intersection occurs).
But Yamamoto as modified does not explicitly disclose determining whether a view of the first annotation menu is occluded by the virtual object; and in response to determining that the view of the first annotation menu is occluded by the virtual object, repositioning the first annotation menu such that the first annotation menu is not occluded by the virtual object.
However Chen disclosed a skilled person before the effective filing date of the claimed invention had already known to reposition a blocked annotation to avoid occlusion ([0021]: In some embodiments, annotations that would otherwise be occluded can be offset (e.g., relocated) and linked to their corresponding object, for instance, by a line (e.g., line segment)). A skilled person would have known that the virtual annotation menu is for the user to make a selection. Therefore the virtual menu should not be blocked by any real or virtual object. A virtual annotation is a virtual object too. Since Chen teaches offsetting blocked annotation, it would have been obvious to a PHOSITA to adopt the teaching of Chen and to add the limitation of determining whether a view of the first annotation menu is occluded by the virtual object; and in response to determining that the view of the first annotation menu is occluded by the virtual object, repositing the first annotation menu such that the first annotation menu is not occlude by the virtual object. The motivation is to make the virtual annotation menu viewable to a user for the user to make selection.
Allowable Subject Matter
Claims 1-6 and 15-25 are allowed.
The following is an examiner’s statement of reasons for allowance:
Prior art, either individually or in combination, fails to disclose or render obviousness the limitation of storing, in a session instance, the second data associated with the virtual object, the virtual annotation, and the second displacement: after the session instance is closed, receiving a request to re-open the session instance; presenting the virtual object at the first position responsive to re-opening the session instance; and presenting the virtual annotation at the second displacement from the first position responsive to re-opening the session instance as claimed in independent claims 1 and the limitation of receiving input indicative of movement of the virtual object; and in response to the movement of the virtual object, moving at least one of the first annotation menu or the second annotation menu with the virtual object while maintaining a relative position between the at least one of the first annotation menu or the second annotation menu and the virtual object as claimed in independent Claim 15. Claims 2-6 and 14-25 are allowed due to their dependency on Claim 1/15.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YINGCHUN HE whose telephone number is (571)270-7218. The examiner can normally be reached M-F 8:00-5:00 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YINGCHUN HE/Primary Examiner, Art Unit 2613
Prosecution Insights