DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This Office Action is in response to the Amendment filed on 08/10/2026.
Response to Arguments
3. Applicant’s arguments with respect to the pending and amended claims have been considered but are not persuasive.
The applicant argues that Csendes does not teach “wherein the target gesture is a pinch-and-release gesture and in response to detecting a pinch part of the pinch-and-release gesture, the virtual plane is able to be dragged to a desired position until a release part of the pinch-and-release gesture is performed”.
The examiner disagrees. Csendes discloses pinch-and-release gesture as described in the claimed invention. In an implementation, the processor is configured to track both the user's finger (being used as the interaction element) and a thumb of the user's hand, focusing on a distance between the first origin point (at the tip of the user's finger) and the third origin point (at the tip of the thumb). This distance, referred to as D.sub.finger, may be calculated to assess whether the user has performed an action that signifies an intent to activate the virtual widget. Activation of the given segment is contingent upon D.sub.finger being less than or equal to the first radius of the interaction volume around tip of the user's finger. If D.sub.finger is greater than this first radius, the processor interprets this as a lack of activation, meaning the user has not performed the necessary action to activate the virtual widget. However, if D.sub.finger is less than or equal to the first radius, this indicates that the user's thumb and the user's finger have come into sufficient proximity to each other, and the virtual widget is activated. Column 12, lines 1-17).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
4. Claims 1-6, 8-12 and 19-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Csendes et al (US 12260062 B1).
Csendes et al (“Csendes”) is directed to Digital Manipulation Of Virtual User Interface Via Activation Of Invisible Virtual Widget.
As per claim 1. Csendes discloses a method (flowchart of Fig. 6) for providing a virtual plane, applied to a host (a display apparatus 100, Fig. 1), comprising:
tracking, by the host, a hand gesture of a hand and determining, by the host, whether the hand has performed a target gesture (a tracking means is responsible for detecting the position, orientation, and movement of the interaction element, such as a user's hand or a specialized pointer (as discussed later), in relation to the virtual user interface, column 5, lines 4-19);
in response to determining that the hand has performed the target gesture, providing, by the host, the virtual plane at a reference height in a virtual world of a reality service (When it is determined that the interaction element is in proximity to the given segment of the virtual widget, the processor is configured to control the at least one light source to display the given segment in the three-dimensional space. That is, after the processor processes the tracking data collected by the at least one tracking means and determines the proximity of the interaction element to the given segment of the virtual widget, a transition occurs in the operation of the display apparatus. Column 6, line 60- column 7, lines 13);
wherein the target gesture is a pinch-and-release gesture and in response to detecting a pinch part of the pinch-and-release gesture, the virtual plane is able to be dragged to a desired position until a release part of the pinch-and-release gesture is performed (In an implementation, the processor is configured to track both the user's finger (being used as the interaction element) and a thumb of the user's hand, focusing on a distance between the first origin point (at the tip of the user's finger) and the third origin point (at the tip of the thumb). This distance, referred to as D.sub.finger, may be calculated to assess whether the user has performed an action that signifies an intent to activate the virtual widget. Activation of the given segment is contingent upon D.sub.finger being less than or equal to the first radius of the interaction volume around tip of the user's finger. If D.sub.finger is greater than this first radius, the processor interprets this as a lack of activation, meaning the user has not performed the necessary action to activate the virtual widget. However, if D.sub.finger is less than or equal to the first radius, this indicates that the user's thumb and the user's finger have come into sufficient proximity to each other, and the virtual widget is activated. Column 12, lines 1-17); and
displaying, by the host, a height adjustment element in the virtual world, wherein the height adjustment element is used for adjusting a height of the virtual plane in the virtual world (The given segment of the virtual widget, which may include elements such as the virtual border of the virtual user interface or other interface components, is thus selectively displayed in response to the user's interaction. This selective visibility enhances the user's focus and interaction by displaying only the relevant parts of the virtual user interface as needed. Furthermore, as the user interacts with the virtual user interface, the processor continually adjusts the display based on the changing position and movement of the interaction element. This adaptive display mechanism ensures that the user's experience is intuitive and seamless, with the virtual user interface responding dynamically to their actions within the three-dimensional space. column 7, lines 14-26, also see column 16, lines 19-36).
As per claim 2, Csendes further discloses that the method according to claim 1, further comprising:
in response to determining that a first distance between the hand and the height adjustment element is smaller than a first distance threshold, changing a first visual type of the height adjustment element to a first type (upon determining the proximity of the interaction element to the given segment of the virtual widget, the processor sends specific commands to the light source to illuminate the given segment of the virtual widget that the interaction element is approaching or engaging with. Column 7, lines 5-9. For moving the panel, the initial step involves the user positioning their finger or the user-interaction controller in close proximity to a side of the panel. Once the user's finger or the pointer of the user-interaction controller is near the side of the panel, the processor presents and activates the virtual widget, and then allows the user to freely move the panel within the virtual user interface. The processor tracks the movement of the user's finger or the user-interaction controller and correspondingly adjusts the position of the panel in real-time. Column 14, lines 46-59).
As per claim 3, Csendes further discloses that the method according to claim 2, further comprising:
in response to determining that the hand has triggered the height adjustment element, changing the first visual type of the height adjustment element to a second type (Further, in an implementation, in the case of the virtual widget being located covering both one of the corner and one of the sides of the virtual user interface, the length of the involved sections would determine the function of the visual effect. In particular, if the length of the side (move section) is greater than the length of the corner (scale section), the processor interprets the user's action as an intent to move the panel. Consequently, the processor initiates the movement effect, causing the virtual user interface panel to reposition within the three-dimensional space in response to the user's input. Conversely, if the length of the corner is greater than the length of the side, the processor interprets the user's action as an intent to resize the panel. Consequently, the processor initiates the resizing effect, adjusting the size of the virtual user interface panel in response to the user's input. Column 16, lines 37-52).
As per claim 4, Csendes further discloses that the method according to claim 1, wherein the virtual plane is displayed with a predetermined width and a predetermined length (Referring to FIGS. 1, 2 and 3A-3B, in combination, in the display apparatus 100, the at least one processor 130 is configured to control the at least one light source 110 to display the virtual user interface 200 in the three-dimensional space 300 (see FIGS. 1, 2 and 3A-3B). In particular, a height, a width, and a depth of the virtual user interface directly influence the proportional scaling of each segment, ensuring that the segments are appropriately sized relative to the entire interface, column 15, lines 62-column 16, lines 3).
As per claim 5, Csendes further discloses that the method according to claim 1, further comprising:
providing a size adjustment element, wherein the size adjustment element is used for adjusting a size of the virtual plane (In hand tracking, this is done by moving the pointer finger, and with the controller, by manipulating the controller itself. The processor responds to these movements by dynamically adjusting the size of the panel within the virtual user interface. Column 14, lines 46-column 15, lines 3);
in response to determining that a second distance between the hand and the size adjustment element is smaller than a second distance threshold, changing a second visual type of the height adjustment element to a third type (Furthermore, as the user interacts with the virtual user interface, the processor continually adjusts the display based on the changing position and movement of the interaction element. This adaptive display mechanism ensures that the user's experience is intuitive and seamless, with the virtual user interface responding dynamically to their actions within the three-dimensional space, column 7, lines 20-26. In particular, a height, a width, and a depth of the virtual user interface directly influence the proportional scaling of each segment, ensuring that the segments are appropriately sized relative to the entire interface. This proportional relationship ensures that the segments are neither too large, which could lead to inaccurate interactions, nor too small, which may make it difficult to interact with accurately, column 15, lines 62-column 16, lines 6); and
in response to determining that the hand has triggered the size adjustment element, changing the second visual type of the size adjustment element to a fourth type (In an embodiment, the visual effect associated with the given segment of the virtual widget comprises one of: a resizing effect, a movement effect. That is, the visual effect associated with the activation of the given segment of the virtual widget is characterized by specific types of responses, such as, but not limited to, either the resizing effect or the movement effect. These effects are executed by the processor and are displayed via the light source. Herein, when the visual effect associated with the activated segment is the resizing effect, the virtual user interface undergoes a change in size or scale in response to the user's interaction, column 14, lines 24-34).
As per claim 6, Csendes further discloses that the method according to claim 1, wherein the reference height is a height of the hand detected to be performing the target gesture (upon the activation of the given segment, the processor begins to process the tracking data received from the tracking means. This data includes information about the position and movement of the interaction element, which can be a user's finger, a pointer, or any other designated object for interaction. The processor analyzes this data to detect any changes in the position of the interaction element from the moment of activation. Column 13, lines 9-26, Also see column 16, lines 19-36, and column 19, lines 17-39).
As per claim 8, Csendes further discloses that the method according to claim 1, wherein before determining whether the hand has performed the target gesture, the method further comprises:
in response to determining that the hand has performed an L-shaped gesture, accordingly determining a first direction, a second direction, and a reference angle (As illustrated in FIG. 4C, the interaction element 310 is depicted as having made contact with a given segment 212b of the virtual user interface 200, at which point the at least one processor 130 controls the at least one light source 110 to display a visual cue (by prominently displaying the given segment 212b) indicative of the activation of the given segment 212b, column 20, lines 51-57. Also see column 12, lines 1-8, and also see Figs. 8A-8C);
displaying a virtual rectangular area comprising a plurality of corners in the virtual world, wherein one of the corners of the virtual rectangular area is aligned with the reference angle, the virtual rectangular area has a first side extending toward the first direction from the reference angle by a first predetermined length, the virtual rectangular area has a second side extending toward the second direction from the reference angle by a second predetermined length, and a height of the virtual rectangular area in the virtual world corresponds to the L-shaped gesture (Referring to FIGS. 8A-8C, illustrated are different depictions of the virtual user interface 200 being digitally manipulated via the interaction element 310 in accordance with the visual effect associated with the activated given segment 212b, in accordance with embodiments of the present disclosure. In the illustrated examples, the interaction element 310 is a finger of a user. As illustrated in FIG. 8A, the interaction element 310 is positioned in proximity to the virtual user interface 200, ready to initiate interaction for activating the given segment 212b (of the virtual widget 200). As illustrated in FIG. 8B, the interaction element 310 activates the given segment 212b, configuring the at least one processor 130 to apply the visual cue indicative of this activation by displaying the given segment 212b. As illustrated in FIG. 8C, the interaction element 310 has moved, which corresponds to a resizing of the virtual user interface 200, driven by a resizing effect (as the visual effect), represented by reference numeral 800, associated with the given segment 212b. Herein, post-activation, the position of the interaction element 310 has changed, and this change is processed by the at least one processor 130 to digitally manipulate the virtual user interface 200 in accordance with the visual effect associated with the activated given segment 212b, column 21, lines 60-column 22, lines 16, also see column 12, lines 1-8).
As per claim 9, Csendes further discloses that the method according to claim 8, wherein providing the virtual plane at the reference height in the virtual world of the reality service (Referring to FIGS. 3A and 3B, illustrated are exemplary depictions of the virtual user interface 200 in a three-dimensional space 300, in accordance with embodiments of the present disclosure. As illustrated, herein, an interaction element 310 is interacting with a given segment 212b, 214b of the virtual widget 210 of the virtual user interface 200. In the illustrated examples, the interaction element 310 is a finger of a user. In other examples, the interaction element 310 may be a pointer of a user-interaction controller, column 19, lines 43-column 20, lines 5), comprises:
displaying the virtual rectangular area as the provided virtual plane in the virtual world. Referring to FIGS. 1, 2 and 3A-3B, in combination, in the display apparatus 100, the at least one processor 130 is configured to control the at least one light source 110 to display the virtual user interface 200 in the three-dimensional space 300.
As per claim 10, Csendes further discloses that the method according to claim 1, further comprising: in response to determining that the hand has performed a confirmation gesture, fixing a size and position of the virtual plane in the virtual world (FIGS. 8A-8C are illustrations of the virtual user interface being digitally manipulated via the interaction element in accordance with a resizing effect associated with a given segment, with the interaction element being the finger of the user, in accordance with embodiments of the present disclosure; column 2, lines 44-49. Also see Column 14, lines 24-45).
As per claim 11, Csendes further discloses that the method according to claim 10, wherein after fixing the size and position of the virtual plane in the virtual world, further comprising: displaying at least one virtual object in the virtual world, wherein each of the at least one virtual object has a fixed relative position with the virtual plane (The given segment of the virtual widget, which may include elements such as the virtual border of the virtual user interface or other interface components, is thus selectively displayed in response to the user's interaction. This selective visibility enhances the user's focus and interaction by displaying only the relevant parts of the virtual user interface as needed. Furthermore, as the user interacts with the virtual user interface, the processor continually adjusts the display based on the changing position and movement of the interaction element. This adaptive display mechanism ensures that the user's experience is intuitive and seamless, with the virtual user interface responding dynamically to their actions within the three-dimensional space. Column 7, lines 14-26, Following the display of the visual cue, the position of the interaction element, such as the user's finger or a controller's pointer, becomes changeable. The processor is configured to track the changes in the position of the interaction element and process them to digitally manipulate the virtual user interface accordingly. As discussed, the digital manipulation of the virtual user interface is based on the visual effect associated with the activated segment. This ability of the processor to control the light source to display the visual cue upon activation, followed by its capability to digitally manipulate the virtual user interface based on the subsequent movements of the interaction element, ensures a highly responsive and user-centric interface by providing real-time visual feedback to the user, column 14, lines 10-23, and also see column 19, lines 17-32).
As per claim 12, Csendes further discloses that the method according to claim 11, wherein the at least one virtual object comprises a first virtual screen (see at least virtual screens of FIGS. 3A-3B), and the method further comprises:
receiving a first video stream of a first screen from a computing device and displaying the first video stream in the first virtual screen (Referring to FIGS. 1, 2 and 3A-3B, in combination, in the display apparatus 100, the at least one processor 130 is configured to control the at least one light source 110 to display the virtual user interface 200 in the three-dimensional space 300, column 20, lines 6-10).
As per system (host) claim 19, the claim recites similar limitations as that of method claim 1. Thus, claim 19 is also rejected under similar citations given to the method claim 1.
As per a non-transitory computer readable storage medium claim 20, the claim recites similar limitations as that of method claim 1. Thus, claim 20 is also rejected under similar citations given to the method claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claims 13-18 are rejected under 35 U.S.C. 103 as being unpatentable over Csendes et al in view of Terre et al (US 20220229534 A1).
Terre et al (“Terre”) is directed to coordinating cursor movement between a physical surface and a virtual surface.
As per claim 13, although Csendes discloses “display apparatus” that refers to a specialized equipment that is capable of at least displaying a video stream. The video stream is to be presented to a user of the at least one display apparatus. Csendes, however, does not seem to teach wherein the at least one virtual object further comprises a second virtual screen, and the method further comprises: receiving a second video stream of a second screen from the computing device and displaying the second video stream in the second virtual screen.
Terre, on the other hand discloses ([0443] The wearable extended reality appliance may display a virtual cursor 4216 to user 100. Virtual cursor 4216 and a plurality of virtual objects (e.g., virtual screen 112 and virtual widgets 114C, 114D) may be located on a first virtual plane 4210. Virtual widget 114E may appear on a top surface of table 102. A second virtual plane 4212 may overlie the top surface of table 102. First virtual plane 4210 (or an extension of the first virtual plane) may traverse (e.g., intersect with) second virtual plane 4212 (or an extension of the second virtual plane) by a line of intersection 4214. In some examples, first virtual plane 4210 and/or second virtual plane 4212 may be displayed to user 100 by the wearable extended reality appliance. In some examples, first virtual plane 4210 and/or second virtual plane 4212 may not be displayed to user 100.also see [0444]).
Before effective filling date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the teaching of Terre with Csendes so that users of Csendes would be able to interact with one or more screen plane concurrently.
Therefore, it would have been obvious to combine Terre with Csendes to obtain the invention as specified in claim 13.
As per claim 14, Csendes in view of Terre further discloses that the method according to claim 12, wherein after fixing the size and position of the virtual plane in the virtual world, further comprising: displaying a tool bar in the virtual plane, wherein the tool bar comprises a control element (Terre, [0247] With reference to FIG. 21, a user interface 2110 is presented within an exemplary extended reality environment from the perspective of a user of the extended reality environment. The user interface 2110 may display to the user virtual content including a plurality of dispersed virtual objects 2112 across the user interface 2110 such that the user may interact with the plurality of dispersed virtual objects 2112. Also see [0248]).
As per claim 15, Csendes in view of Terre further discloses that the method according to claim 14, wherein the control element is used to control the computing device (Terre, [0341] In some embodiments, the user of the wearable extended reality appliance may interact with the interactive element by manipulating a virtual control in the interactive element; by manipulating a physical control on the wearable extended reality appliance; or by manipulating a physical control on a device in communication with the wearable extended reality appliance, such as an interactive computational interface device.
As per claim 16, Csendes in view of Terre further discloses that the method according to claim 14, wherein the control element is used reconfigure the size and position of the virtual plane in the virtual world (Terre, [0164] Consistent with some disclosed embodiments, the selective position control may include docking the plurality of virtual objects to a physical space, and enabling a wearer of the wearable extended reality appliance to walk in the physical space and adjust a position of each virtual object separately. Docking the plurality of virtual objects to a physical space may include maintaining a size, orientation, distance, or any other spatial attribute of the plurality of virtual objects with reference to a physical space. This type of selective position control may allow a user to move within the physical space while maintaining a size, orientation, distance, or any other spatial attribute of the plurality of virtual objects maintained as the user moves). Also see [0300]).
As per claim 17, Csendes in view of Terre further discloses that the method according to claim 10, wherein the virtual plane is a virtual desktop, the hand gesture is tracked during a desktop configuring process of an application of the reality service (Terre, [0249] Some disclosed embodiments may involve receiving an initial kinesics input tending toward the first virtual region. The term kinesics input may relate to a user's natural human movements and/or gestures. Such movements may occur within a real coordinate system which may be tracked and/or captured as input data that may be translated to virtual actions within a virtual coordinate system of the extended reality environment. The kinesics input of the user may include hand gestures, body gestures (e.g., movements of the user's head or body), eye-movements or other eye movements, and/or any other gesture-based interaction, facial movements, or combination of gesture-based interactions. and the method further comprises:
in response to determining that the hand has performed the confirmation gesture, storing the size and position of the virtual plane as a desktop configuration and finishing the desktop configuring process (Terre, [0177] Some embodiments may involve obtaining default settings for displaying content using the wearable extended reality appliance. Default settings may be stored in memory, and may be preset by a manufacturer, distributer, or user. Settings may include color, size, resolution, orientation, or any other conditions associated with a display. Default settings may include any a preselected option for such conditions adopted by a computer program or other mechanism when no alternative is specified at the time of use by a user or programmer. The default settings may be received from a user or determined based on a type of wearable extended reality appliance used, a time, a location, or any other type of heuristic data. Further embodiments of the non-transitory computer readable medium include selecting the first perspective of the scene based on the default settings).
As per claim 18, Csendes in view of Terre further discloses that the method according to claim 17, wherein after finishing the desktop configuring process, the method further comprises: in response to determining that the application has been launched, displaying the virtual plane based on the stored desktop configuration and displaying at least one virtual screen in the virtual world, wherein each of the at least one virtual screen has a fixed relative position with the virtual plane (Terre, [0145] Controlling perspective as used in this disclosure may include, for example, changing a position of a scene in fixed distance, changing a distance of the scene, changing an angle of the scene, changing a size of the scene, causing rotation of the scene, or any other manipulation of the scene that may cause a change in how a viewer may visualize the scene. In one example, controlling perspective may include changing a spatial transformation of the scene. [0439] In some examples, a position, an orientation, or any other aspect of the second virtual plane may be same as or similar to that of the physical surface. In some embodiments, dimensions of the second virtual plane may correspond with dimensions of the physical surface. For example, the physical object and the second virtual plane may share the same or similar dimensions, or may otherwise overlap. For example, the physical object may correspond with dimensions of the second virtual plane in one or more of a position, an orientation, a size, a contour, a form, or any other aspect of a surface.
Conclusion
6. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to TADESSE HAILU whose telephone number is (571)272-4051; and the email address is Tadesse.hailu@USPTO.GOV. The examiner can normally be reached Monday- Friday 9:30-5:30 (Eastern time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bashore, William L. can be reached (571) 272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TADESSE HAILU/ Primary Examiner, Art Unit 2174
Prosecution Insights