DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on/after Mar. 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Arguments
Applicant’s arguments, see p. 12, filed 04 June 2026, with respect to claim 20 have been fully considered and are persuasive. The eligibility rejection of 04 March 2026 has been withdrawn.
Applicant’s arguments, see p. 12-15, filed 04 June 2026, with respect to the rejection(s) of claims 1 and 18-19 under 35 U.S.C. § 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Bernstein et al. (U.S. PG-PUB 2017/0010846). The previously cited TERRANO reference is not relied upon to reject any of the newly amended independent claims. Please see the Office action below for further rationale for the rejection of the newly amended independent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6, 8-10, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gimpl et al. (U.S. PG-PUB 2012/0081319, 'GIMPL') in view of Bernstein et al. (U.S. PG-PUB 2017/0010846, 'BERNSTEIN').
Regarding claim 1, GIMPL discloses an interaction method of a virtual space, comprising:
presenting an interaction navigation panel in the virtual space in response to a wake-up instruction of the virtual space (GIMPL; FIG. 1H; ¶ 0084-85; “In addition to the open position, device 100 may also have a "closed" position … Position sensors 172A and 172B can generate a signal indicating that device 100 is in the "closed" position. This can trigger an event [wake-up instruction] that results in a change of displayed information on screen 104 and/or 108. … device 100 may be programmed to stop displaying information on one of the screens, e.g., screen 108, since a user can only view one screen at a time when device 100 is in the "closed" position.”), wherein the interaction navigation panel comprises at least two pending interaction objects (GIMPL; FIG. 5A; ¶ 0144; “The Application Manager 562 is an application that provides a presentation layer for the window environment. Thus, the Application Manager 562 provides the graphical model for rendering by the Task Management Module 540. … the Desktop 566 provides the presentation layer for the Application Store 560. … the desktop provides a graphical model [‘virtual space’] of a surface [‘interaction navigation panel’] having selectable application icons [‘pending interaction objects’] for the Applications 564 in the Application Store 560 that can be provided to the Window Management Module 556 for rendering.” [The Examiner asserts that GIMPL IMPLIEDLY teaches an interaction navigation panel; however, the Examiner further asserts that BERNSTEIN EXPLICITLY teaches a specific, distinct, and dedicated interaction navigation panel {dynamic function row 104} juxtaposed between a panel display and a keyboard of a conventional laptop computing device, or adjacent to a conventional, peripheral, fixed-key keyboard. In other words, BERNSTEIN teaches an interaction navigation panel that is not also the display screen/panel. Please see further analysis and explanation for the rationale for the rejection of the amended claim language in the Office action below.]);
in response to a triggering operation on an interaction object of the pending interaction objects, determining a first display panel of an interaction page associated with the interaction object (GIMPL; FIGS. 1A-1J; ¶ 0077-78; “Primary screen 104 also includes a configurable area 112 … for specific inputs when the user touches portions of the configurable area 112. Secondary screen 108 also includes a configurable area 116 that … for specific inputs. … Areas 112b and 116b … receive a "menu" input indicating that the user would like to view options from a menu. Areas 112c and 116c [‘interaction objects’] … receive a "home" input [‘triggering operation’] indicating that the user would like to view information associated with a "home" view. In other embodiments, areas 112a-c and 116a-c [‘interaction objects’] may be configured … for other types of specific inputs [‘triggering operations’] including controlling features of device 100, … including … selecting of displayed items [‘interaction object’] (on either of screen 104 [‘determining a first display panel of an interaction page’] or 108) … Also, … areas 112a-C and 116a-C may be … for specific inputs depending upon the application running on device 100 and/or information displayed on touch sensitive displays 110 [‘first display panel’] and/or 114.”);
PNG
media_image1.png
297
329
media_image1.png
Greyscale
in response to the first display panel is a close/long-range panel, waking up the close/long-range panel in the virtual space and displaying the interaction page associated with the interaction object on the close/long-range panel (GIMPL; FIGS. 7A-7B; ¶ 0167; “The device 100 manages desktops and/or windows with … window stack 700 … [which] is a logical arrangement of active and/or inactive windows or display objects for a multi-screen device. … the window stack 700 may be logically like a deck of cards or a stack of bricks, where … window(s) or display object(s) (e.g., desktop(s)) are arranged in an order … An active window is a window that is currently being displayed [‘waking up the close-range panel in the virtual space’] on … touch sensitive display(s) 110, 114. … Window 2 712 is an active window and displayed on only one of the touch sensitive displays 114. In … FIG. 7A, the device 100 is in a closed state 304 displaying window 2 712 in configuration 608. An inactive window is a window that was opened and displayed but is now "behind” [long-range panel’] an active window and not being displayed. … an inactive window may be for an application that is suspended, and thus, the window is not displaying active content. … window 1 708 and window 3 716 are inactive windows [‘long-range panels’].” FIGS. 7C-7J; ¶ 0168-179; [The Examiner notes that GIMPL teaches numerous ways to stack/prioritize various windows, and to display them on an arbitrary display panel that may be aside/behind a first/primary display panel.] ¶ 0196; “… in one alternative embodiment, the window stack 700 resembles a carousel or rolodex rather than a deck of cards. Thus, windows may circulate from one touch sensitive display 110 to another touch sensitive display 114. … [windows] may be [pushed] to the right and end up at the end of stack behind another window. If the stack continues to move to the right, eventually the window would appear on the second touch sensitive display [‘waking up the long-range panel in the virtual space and displaying the interaction page associated with the interaction object on the long-range panel’] even though the window was opened in the first touch sensitive display 110.”), wherein
PNG
media_image2.png
393
734
media_image2.png
Greyscale
the close/long-range panels … display independently and in different positions (GIMPL; FIGS 1A-1B; ¶ 0077; “Primary screen 104 includes touch sensitive display 110, which, in addition to being touch sensitive, also displays information to a user. Secondary screen 108 includes touch sensitive display 114, which also displays information to a user.” FIG. 1C; ¶ 0080; “The two screens 104 and 108 are connected with a hinge 128 … (illustrating a back view of device 100). Hinge 128 … is a center hinge that connects screens 104 and 108 so that when the hinge is closed, screens 104 and 108 are juxtaposed (i.e., side-by-side) as shown in FIG. 1B (illustrating a front view of device 100). Hinge 128 can be opened to position the two screens 104 and 108 in different relative positions to each other. … the device 100 may have different functionalities depending on the relative positions of screens 104 and 108.” [The Examiner notes that ‘screen 108’ is positioned rearwardly from ‘screen 104’; the Examiner asserts that, in … the configuration depicted in FIG. 1A of GIMPL, ‘screen 108’ is analogous to a long-range panel that displays independently and [is] in [a] different position from ‘screen 104’.]), and wherein the
at least two pending interaction objects are on a same interaction navigation panel (GIMPL; FIGS 1A-1B; ¶ 0078; “Primary screen 104 also includes a configurable area 112 … for specific inputs when the user touches portions of the configurable area 112. Secondary screen 108 also includes a configurable area 116 … for specific inputs.” [The Examiner asserts that the ‘configurable areas 112/116’ of FIGS. 1A-1B of GIMPL IMPLIEDLY teach a same interaction navigation panel having at least two pending interaction objects, in that there are at least three buttons to be interacted with apart and distinct from the main touch screens displays. The Examiner, for expediency and thoroughness, further asserts that BERNSTEIN EXPLICITLY teaches that at least two pending interaction objects are on the same interaction navigation panel, as explained below.]).
PNG
media_image3.png
332
407
media_image3.png
Greyscale
GIMPL does not EXPLICITLY disclose that at least two pending interaction objects are on a same interaction navigation panel, which BERNSTEIN EXPLICITLY discloses (BERNSTEIN; FIG. 1B; ¶ 0040; “Body portion 120 include a set of physical keys 106, a dynamic function row 104 [same interaction navigation panel], and a touch pad 108 partially contained within a same housing. … dynamic function row 104, which is a touch screen, replaces a function row of the set of physical keys 106 allowing the space consumed by the set of physical keys 106 to be reduced, allowing for a smaller overall body portion 120 or allowing other portions, such as touch pad 108, to be larger.” ¶ 0043; “… dynamic function row 104 is a touch screen display that displays … user-selectable symbol(s) 142 [at least two pending interaction objects] (… herein called “[UI] elements,” “[UI] components,” “affordances,” “buttons,” or “soft keys”). … dynamic function row 104 replaces the function row keys on a typical keyboard. A user may select a particular one of the … user-selectable symbol(s) 142 by touching a location on the touch screen display that corresponds to the … one of the … user-selectable symbol(s) 142.” ¶ 0046; “Each of the … user-selectable symbol(s) 142 may include an indicator, such as a symbol (e.g., a magnifying glass symbol as shown at 144), an abbreviated word (e.g., “esc”), … a character, an image, an animated image, a video, … A respective one of the … user-selectable symbol(s) 142 … [receives] user input(s).”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the interaction method of a virtual space of GIMPL to include the disclosure that at least two pending interaction objects are on a same interaction navigation panel of BERNSTEIN. The motivation for this modification is to address the shortcoming of requiring the user to switch modes and move the location of his/her hands between keyboard and mouse by providing dynamic and space efficient I/O devices and methods, which optionally complement or replace conventional input devices and methods, reduce the amount of mode switching (e.g., moving one's hands between keyboard and mouse, and also reduce the amount of moving one's eyes from keyboard to display) required of a user and produce a more efficient human-machine interface (BERNSTEIN; ¶ [0004-5]).
Regarding claim 18, GIMPL-BERNSTEIN discloses an interaction apparatus of a virtual space, comprising:
a first response module for … ([Please see the treatment of this parallel limitation above.]);
a second response module for determining a target display panel of an interaction page associated with an interaction object in response to a triggering operation (BERNSTEIN; FIG. 3A; ¶ 0070-71 “… display system 312 (e.g., primary display 102 [target display panel] of display portion 110, FIG. 1A and/or dynamic function row 104, FIGS. 1A-1B) is a touch-sensitive display with a touch-sensitive surface … that accepts input from the user based on … tactile contact [triggering operation]. … display system 312 and display controller 356 (along with any associated modules [second response module] and/or sets of instructions in memory 302) detect contact (and any movement or breaking of the contact) on display system 312 and convert the detected contact into interaction with [UI] objects (e.g., … soft key(s), icon(s), web page(s), or image(s)) that are displayed on display system 312.”);
a first display module for … ([Please see the treatment of this parallel limitation above.]);
a second display module for … ([Please see the treatment of this parallel limitation above.]) if the target display panel is the long-range panel (GIMPL; FIG. 1A; [The Examiner notes that screen 108 is positioned rearwardly relative to screen 104; thus, the Examiner asserts that screen 108 is analogous to a long-range panel.]; FIG. 2, ‘screen 108’; ¶ 0093; “… display controllers 216a/216b [are] provided for controlling the operation of the touch sensitive screens 104/108, including input (touch sensing) and output (display) functions. … a separate touch screen controller 216a/216b is provided for each touch screen 104 and 108.”); and …
([Please see the treatment of this parallel limitation above.]).
Regarding claim 19, GIMPL-BERNSTEIN discloses an electronic device, comprising:
a processor and a memory, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs stored in the memory to execute an interaction method of a virtual space (GIMPL; FIG. 2, ‘processor 204’ and ‘memory 208’; ¶ 0094-95; “A communication device 100 may also include memory 208 for use in connection with the execution of application programming or instructions by the processor 204, and for the temporary or long term storage of program instructions and/or data.”), wherein the interaction method of the virtual space comprises: … ([The remaining limitations are repeated verbatim from those recited in claim 1.]).
Regarding claim 2, GIMPL-BERNSTEIN discloses the method according to claim 1, wherein the determining the first display panel of the interaction page associated with the interaction object comprises:
PNG
media_image4.png
386
601
media_image4.png
Greyscale
determining a type of the interaction object; and
determining the first display panel of the interaction page associated with the interaction object based on the type of the interaction object (BERNSTEIN; FIG. 5Y; ¶ 0192-193; “FIG. 5Y illustrates primary display 102 displaying a window 5166 for the photo application [‘determining the first display panel of the interaction page’] in response to detecting selection of photo application icon 515 [‘associated with the interaction object based on the type of the interaction object’] … Window 5166 displays … photos associated with an all-photos sub-section of a user's photo library. … the all-photos sub-section of a user's photo library is in focus on primary display 102 [‘determining the first display panel of the interaction page’] as shown by “Photos” … Status tray 502 indicates that the photo application is running in the foreground, and app tray 514 also indicates that the photo application is running in the foreground based on the shadow behind photo application icon 515. … FIG. 5Y further illustrates dynamic function row 104 [‘interaction navigation panel’] displaying … affordances corresponding to the all photos sub-section of the user's photo library (e.g., affordances 5168, 5170, and 5172) and the … system-level affordance(s) (e.g., affordance 542) in response to detecting selection of photo application icon 515 … Dynamic function row 104 includes: search affordance 5168, for searching the user's photo library; slideshow affordance 5170, which, when activated (e.g., via a tap contact), initiates a slideshow of the selected photos or all photos in the all photos sub-section of the user's photo library in a full-screen mode …; and slider affordance 5172 for scrolling the photos displayed from the all photos sub-section of the user's photo library that are displayed in window 5166.”).
Regarding claim 3, GIMPL-BERNSTEIN discloses the method according to claim 2, wherein the determining the type of the interaction object comprises:
obtaining identification information of the interaction object; and
determining the type of the interaction object based on the identification information (BERNSTEIN; ¶ 0192; “FIG. 5Y illustrates primary display 102 displaying a window 5166 for the photo application in response to detecting selection of photo application icon 515 with cursor 504 …”).
Regarding claim 4, GIMPL-BERNSTEIN discloses the method according to claim 2, wherein the determining the first display panel of the interaction page associated with the interaction object based on the type of the interaction object, comprises:
searching for the first display panel of the interaction page associated with the interaction object (BERNSTEIN; ¶ 0192; “FIG. 5Y illustrates primary display 102 [‘first display panel’] displaying a window 5166 [‘interaction page’] for the photo application in response to detecting selection of photo application icon 515 [‘interaction object’] with cursor 504 …”) in a mapping relationship between an interaction object type and a display panel based on the type of the interaction object (BERNSTEIN; ¶ 0193; “… FIG. 5Y further illustrates dynamic function row 104 displaying … affordances corresponding to the all-photos sub-section of the user's photo library (e.g., affordances 5168, 5170, and 5172) and the … system-level affordance(s) (e.g., affordance 542) in response to detecting selection of photo application icon 515”).
Regarding claim 6, GIMPL-BERNSTEIN discloses the method according to claim 1, wherein the interaction page comprises a first interaction control, and the method further comprises:
presenting a virtual input model in the virtual space in response to a triggering operation on the first interaction control (BERNSTEIN; ¶ 0192; “FIG. 5Y illustrates primary display 102 displaying a window 5166 for the photo application [‘presenting a virtual input model in the virtual space’] in response to detecting selection of photo application icon 515 [‘in response to a triggering operation on the first interaction control’] …”); and
presenting corresponding input interaction information on the interaction page based on an input operation by a user acting on the virtual input model (BERNSTEIN; FIG. 5Y; ¶ 0192-193; “Window 5166 displays … photos associated with an all-photos sub-section of a user's photo library. … the all-photos sub-section of a user's photo library is in focus on primary display 102 as shown by “Photos” displayed in bold and photos A-L at least partially displayed in window 5166. FIG. 5Y further illustrates dynamic function row 104 displaying … affordances corresponding to the all-photos sub-section of the user's photo library (e.g., affordances 5168, 5170, and 5172) and the … system-level affordance(s) (e.g., affordance 542) in response to detecting selection of photo application icon 515 …”).
Regarding claim 8, GIMPL-BERNSTEIN discloses the method according to claim 6, wherein the presenting the corresponding input interaction information on the interaction page based on the input operation by the user acting on the virtual input model, comprises:
PNG
media_image5.png
384
598
media_image5.png
Greyscale
displaying corresponding text (BERNSTEIN; ¶ 0175-176; “FIG. 5M illustrates primary display 102 displaying a sub-window for composing a new email within window 580 [‘interaction region of the interaction page’] in response to detecting selection of the email composition affordance with cursor 504 … In FIG. 5M, the sub-window for composing a new email is in focus on primary display 102 … FIG. 5M also illustrates dynamic function row 104 displaying … affordances corresponding to composing a new email in response to detecting selection of the email composition affordance … The … affordances corresponding to composing a new email includes … affordance 5116 for changing the text color of selected the text of the new email; affordance 5118 for emboldening selected text of the new email; affordance 5120 for italicizing selected text of the new email; and affordance 5122 for underlining selected text of the new email [‘text … input operation acting on the virtual input model’].”);
Regarding claim 9, GIMPL-BERNSTEIN discloses the method according to claim 6, wherein
the virtual input model comprises an input region and a display region (GIMPL; FIG. 1A-1B; ¶ 0077; “Primary screen 104 includes touch sensitive display 110, which, in addition to being touch sensitive, also displays information to a user. … screens 104/108 may include [several] display areas.”; ¶ 0078; “Primary screen 104 also includes a configurable area 112 that has been configured for specific inputs when the user touches portions of the configurable area 112.”);
the presenting the corresponding input interaction information on the interaction page based on the input operation by the user acting on the virtual input model, comprises:
PNG
media_image6.png
389
602
media_image6.png
Greyscale
displaying the corresponding input interaction information in the display region based on an input operation by the user acting on the input region (BERNSTEIN; FIG. 5J; ¶ 0170; “… primary display 102 displays a window 580 for the mail application in response to detecting selection of mail application icon 506 … Window 580 displays a list of … emails (e.g., emails A-F) [corresponding input interaction information] in a user's inbox and the contents of … email A.”); and
displaying the corresponding input interaction information on the interaction page in response to a triggering operation on a sending button in the input region (BERNSTEIN; ¶ 0185; “FIG. 5S illustrates primary display 102 displaying “James H.” in the “To:” field of the sub-window for composing a new email within window 580 in response to detecting selection of contact affordance 5146-D in FIG. 5R [triggering operation on a sending button in the input region]. FIG. 5S also illustrates dynamic function row 104 replacing display of the … contact affordances 5146 (e.g., contact affordances 5146-A to 5146-F) corresponding to the “Family” group of the user's contact book with the … affordances (e.g., affordance 5114 [¶ 0176: ‘which, when activated …, causes … system 100/200 to send the newly composed email …’] …) corresponding to composing a new email in response to detecting selection of contact affordance 5146-D in FIG. 5R.”).
Regarding claim 10, GIMPL-BERNSTEIN discloses the method of claim 6, wherein before presenting the virtual input model in the virtual space, the method further comprises:
hiding the interaction navigation panel presented in the virtual space (BERNSTEIN; ¶ 0251; “… after displaying the first set of … affordance(s) for a first predetermined period of time …, dynamic function row 104 turns off until a contact is detected by dynamic function row 104 or a keyboard or touch pad associated with the computing system.”).
Regarding claim 20, GIMPL-BERNSTEIN disclose a non-transitory computer-readable storage medium for storing computer programs which cause a computer to execute the interaction method of the virtual space (GIMPL; FIG. 9; ¶ 0186; “… a method 900 for changing a window stack [‘interaction method of the virtual space’] is shown … The method 900 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer readable medium. … the method 900 shall be explained with reference to the systems, components, modules, software, data structures, user interfaces, etc. …”) according to claim 1 ([See analysis of claim 1 above.]).
Claims 5 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over GIMPL in view of BERNSTEIN as applied to claims 4 and 1 above, respectively, and further in view of Terrano (U.S. PG-PUB 2020/0090401, 'TERRANO').
Regarding claim 5, GIMPL-BERNSTEIN discloses the method according to claim 4; however, GIMPL-BERNSTEIN do not explicitly disclose that the method according to claim 4 further comprises the following limitation(s), which TERRANO discloses:
in response to the first display panel is not found (TERRANO; FIG. 6; ¶ 0052; “The virtual panel having the second opacity may allow the user to see through the virtual panel. … the virtual panel with the second opacity may be transparent and the virtual panel may be invisible to the user. The virtual panel may disappear from the view of the user [target display panel is not found] when the system changes the virtual panel's opacity to the second opacity. As another example, the virtual panel may be translucent and may allow the user the see through.”), determining the long-range panel as the first display panel of the interaction page associated with the interaction object according to a predetermined display rule (TERRANO; FIG. 3B; ¶ 0046; “… the system 300 may determine that the vergence distance of the user 360 is beyond the virtual panels (e.g., 310, 320) for a threshold distance [‘predetermined display rule’] and the virtual panels 310 and 320 are at least partially within the view of the user looking at the TV 380 [‘determining the long-range panel as the target display panel of the interaction page’].”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method according to claim 4 of GIMPL-BERNSTEIN to include the in response to the first display panel is not found, determining the long-range panel as the first display panel of the interaction page associated with the interaction object according to a predetermined display rule of TERRANO. The motivation for this modification is to ensure that the user is constantly always presented with at least one display panel, which ensures confidence within the user that the interaction page is constantly presenting useful information and is functioning properly.
Regarding claim 13, GIMPL-BERNSTEIN discloses the method according to claim 1; however, GIMPL-BERNSTEIN do not explicitly disclose that after displaying the interaction page associated with the interaction object on the close-range panel, the method further comprises:
in response to any other interaction object in the interaction navigation panel is detected to be triggered and a first display panel of an interaction page associated with the other interaction object is the long-range panel, waking up the long-range panel in the virtual space and displaying the interaction page associated with the other interaction object on the long-range panel, which TERRANO discloses (TERRANO; ¶ 0046; “FIG. 3B illustrates an example usage of an artificial reality system 300 by a user 360 watching a TV 380 [‘long-range panel’] behind virtual display panels. The user 360 may use the virtual panels (e.g., 310-340) for reading or working activities. The user 360 may … watch a TV 380 which is partially behind the virtual panels 320 and partially behind the virtual panel 310. System 300 may determine the vergence distance and gazing point of the user 360 using an eye tracking system. As soon as the user 360 moves his eyes from the virtual panels to the TV 380, the system 300 may detect that as an indication of the user 360 to look through the virtual panels (e.g., 310, 320) that interfere with the view of the user 360. … the system 300 may determine that the vergence distance of the user 360 is beyond the virtual panels (e.g., 310, 320) for a threshold distance and the virtual panels 310 and 320 are at least partially within the view of the user looking at the TV 380. The system 300 may change the opacity of the virtual panels 310 and 320 into transparent or translucent to allow the user to see through them.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method according to claim 1 of GIMPL-BERNSTEIN to include the waking up the long-range panel in the virtual space and displaying the interaction page associated with the other interaction object on the long-range panel of TERRANO. The motivation for this modification is to enable a user to focus on a background window that is occluded by a closer window, and to further enable the user to bring the background window into the foreground using the user’s gaze.
Regarding claim 14, GIMPL-BERNSTEIN discloses the method according to claim 1; however, GIMPL-BERNSTEIN do not explicitly disclose that after displaying the interaction page associated with the interaction object on the long-range panel, the method further comprises:
in response to any other interaction object in the interaction navigation panel is detected to be triggered and a first display panel of an interaction page associated with the other interaction object is the close-range panel, waking up the close-range panel in the virtual space and displaying the interaction page associated with the other interaction object on the close-range panel, which TERRANO discloses (TERRANO; ¶ 46; “When the user 360 moves his eyes from the TV 380 back to the virtual panel 310 or 320 [‘close-range panels’], the user 360 just needs to focus on his eyes on the visual anchor 318 or 324. System 300 may detect that the user is looking at the visual anchor 318 or 324 and change the corresponding panel 310 or 320 back to visible. The virtual panels 330 and 340 may or may not change since they are not interfering with the user view for watching the TV 380.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method according to claim 1 of GIMPL-BERNSTEIN to include the waking up the close-range panel in the virtual space and displaying the interaction page associated with the other interaction object on the close-range panel of TERRANO. The motivation for this modification is to enable a user to focus on a foreground window that is occluded by a farther, opaquer window using the user’s gaze.
Claims 7 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over GIMPL in view of BERNSTEIN as applied to claim 6 above, and further in view of Regenbrecht et al. ("A Tangible AR Desktop Environment", published 2001, 'REGENBRECHT').
Regarding claim 7, GIMPL-BERNSTEIN discloses the method according to claim 6; however, GIMPL-BERNSTEIN do not explicitly disclose that the presenting the virtual input model in the virtual space comprises:
PNG
media_image7.png
327
721
media_image7.png
Greyscale
in response to the interaction page is displayed on the close/long-range panel, presenting a close/long-range virtual input model corresponding to the close/long-range panel in the virtual space, which REGENBRECHT discloses (REGENBRECHT; FIG. 7; [The Examiner notes that the figure depicts three windows, such as those that are presented in Microsoft Windows, both as a conventional 2-D rendering and as a three-dimensional rendering using an HMD. The Examiner notes that the central window is placed rearwardly, while the windows depicting the file system and the model of the car are placed closer to the viewer.]), wherein the close-range virtual input model and the long-range virtual input model are displayed independently and in different positions (REGENBRECHT; FIG. 7; [The Examiner notes that the three windows are analogous to the close/long-range virtual input models, which are independent/distinct windows and are placed differently throughout the 3-D desktop space.]).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method according to claim 6 of GIMPL-BERNSTEIN to include the disclosure that if the interaction page is displayed on the close/long-range panel, presenting a close/long-range virtual input model corresponding to the close/long-range panel in the virtual space, wherein the close-range virtual input model and the long-range virtual input model are displayed independently and in different positions of REGENBRECHT. The motivation for this modification is to intuitively simulate the real environment of a human user by inserting 2-D graphics upon his/her actual desktop workspace, bringing the metaphorical computer operating system desktop back into the physical world by superimposing the graphical windows upon the physical desk by augmented reality.
Regarding claim 12, GIMPL-BERNSTEIN-REGENBRECHT discloses the method according to claim 7, wherein in response to the interaction page is displayed on the close-range panel, the presenting the first prompt pop-up window associated with the second interaction control in the virtual space comprises:
displaying the first prompt pop-up window associated with the second interaction control at a first predetermined position between the close-range panel and the interaction navigation panel presented in the virtual space (REGENBRECHT; FIG. 7; [The Examiner notes that the placement of the prompt pop-up windows is arbitrary and based upon the physical objects in the scene. The Examiner asserts that the placement of the prompt pop-up windows is an arbitrary design choice.]); and
in response to the interaction page is displayed on the long-range panel, the presenting the first prompt pop-up window associated with the second interaction control in the virtual space, comprises:
displaying the first prompt pop-up window associated with the second interaction control at a second predetermined position between the long-range panel and the interaction navigation panel presented in the virtual space (REGENBRECHT; FIG. 7; [The Examiner notes that the placement of the placement of the prompt pop-up windows is arbitrary and based upon the physical objects in the scene. The Examiner asserts that the placement of the prompt pop-up windows is an arbitrary design choice.]).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over GIMPL in view of BERNSTEIN as applied to claim 1 above, and further in view of Katz (U.S. PG-PUB 2004/0078240, 'KATZ').
Regarding claim 11, GIMPL-BERNSTEIN discloses the method according to claim 1; however, GIMPL-BERNSTEIN do not explicitly disclose that the interaction page further comprises … presented second interaction control(s); and that the method further comprises [the following features], which KATZ discloses:
PNG
media_image8.png
237
429
media_image8.png
Greyscale
in response to a triggering operation on a second interaction control of the … presented second interaction control(s),
presenting a first prompt pop-up window associated with the second interaction control in the virtual space (KATZ; FIGS. 3-4; ¶ 0046-47), wherein the first prompt pop-up window at least comprises a confirmation sub-control and a cancellation sub-control;
executing an interaction operation associated with the second interaction control in response to a triggering operation on the confirmation sub-control; and
cancelling execution of the interaction operation associated with the second interaction control in response to a triggering operation on the cancellation sub-control (KATZ; FIGS. 3-4; ¶ 47; “… Patient Name pop-up window 402 … is used to personalize the flow sheet to an individual patient. This pop-up window is initiated by clicking on the "change username" 320 drop-down menu item [‘in response to a triggering operation on a second interaction control’] … The patient enters his name at 404 and when satisfied …, clicks on "OK" button 406 [‘confirmation sub-control’] to save the change [‘interaction operation associated with the second interaction control in response to a triggering operation on the confirmation sub-control’] or "Cancel" button 408 [‘cancellation sub-control’] to exit the patient’s name pop-up window without saving changes [‘interaction operation associated with the second interaction control in response to a triggering operation on the cancellation sub-control’].”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method according to claim 1 of GIMPL-BERNSTEIN to include the various teachings of KATZ. The motivation for this modification is to allow a user to either enter textual information by confirming the entry with an ‘OK’ button, or to allow a user to cancel the entry with a ‘CANCEL’ button.
Claims 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over GIMPL in view of BERNSTEIN as applied to claim 1 above, and further in view of Pizano (U.S. PG-PUB 2008/0086646, 'PIZANO').
Regarding claim 15, GIMPL-BERNSTEIN discloses the method according to claim 1; however, GIMPL-BERNSTEIN do not explicitly disclose that the method according to claim 1 comprises:
PNG
media_image9.png
329
412
media_image9.png
Greyscale
displaying a second prompt pop-up window in the virtual space, wherein the second prompt pop-up window is displayed in front of the close-range panel, which PIZANO discloses (PIZANO; FIG. 13; ¶ 0090; “… presenting a pop-up window with text fields for a username and a password … Such a pop-up window may be similar like122 …” ¶ 0094; “While the illustrated window 116 is a word processor, it will be appreciated that substantially any application that is compatible with the host operating environment … may be used …” [The Examiner asserts that ‘window 116’ is analogous to the ‘close-range panel’ of the instant claim(s).]).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method according to claim 1 of GIMPL-BERNSTEIN to include the displaying a second prompt pop-up window in the virtual space, wherein the second prompt pop-up window is displayed in front of the close-range panel of PIZANO. The motivation for this modification is to assert security protocols by superseding the active application to authenticate a permitted user.
Regarding claim 16, GIMPL-BERNSTEIN-PIZANO discloses the method according to claim 15, wherein the displaying the second prompt pop-up window in the virtual space comprises at least one of the following:
displaying an authentication prompt pop-up window in the virtual space in response to a detection of an authentication instruction (PIZANO; ¶ 0081; “… the recipient client 38 identifies the secure package identifier 86 and prompts the recipient to submit identification and authentication information. The recipient client 38 may prompt the user to submit the identification and authentication information … by presenting … pop-up window(s) with text fields for receiving a username and a password …”)
.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over GIMPL in view of BERNSTEIN as applied to claim 1 above, and further in view of Kulewski et al. (U.S. Patent 10,101,891; 'KULEWSKI').
Regarding claim 17, GIMPL-BERNSTEIN discloses the method according to claim 1; however, GIMPL-BERNSTEIN do not explicitly disclose that the method according to claim 1 further comprises:
PNG
media_image10.png
501
440
media_image10.png
Greyscale
adjusting the interaction navigation panel, the close-range panel, and/or the long-range panel presented in the virtual space in response to an adjustment operation on the interaction navigation panel, the close-range panel, and/or the long-range panel, wherein the adjusting the interaction navigation panel, the close-range panel, and/or the long-range panel presented in the virtual space (KULEWSKI; FIG. 7; Col. 18, Lines 56-61; “… method 700 implements block 612 of FIG. 6, in which the crop window is changed to fit within the image boundaries.”), comprises:
in response to the adjustment operation is a scaling adjustment operation, performing scaling adjustment on the interaction navigation panel, the close-range panel, and/or the long-range panel based on the scaling adjustment operation (KULEWSKI; FIG. 7; Col. 18, Lines 62-67 ~ Col. 19, Lines 1-4; “In block 702, it is checked whether the crop window was resized based on the user input. … the resizing can be like the resizing described above with reference to FIGS. 2-3, based on a user moving a vertex or segment of the crop window within the user interface. … the resizing may cause … portion(s) of the crop window to move outside the image boundaries. If a resizing has been performed, the method continues to block 704 in which the crop window is resized [‘scaling adjustment operation’] to be positioned within the image boundaries.”);
in response to the adjustment operation is an orientation adjustment operation, adjusting an orientation of the interaction navigation panel, the close-range panel, and/or the long-range panel based on the orientation adjustment operation (KULEWSKI; Col. 19, Lines 39-50; “In block 710, it is checked whether relative rotation occurred between the crop window and the image based on the user input. … the user input may have rotated the image relative to the crop window or rotated the crop window relative to the image. … user input causing such rotation can be provided by selecting a portion of the image or crop window and moving a pointer (e.g., finger on touchscreen, cursor controlled by pointing device, etc.) to rotate the image or crop window. … the rotation can be caused by user input [‘orientation adjustment operation’] such as commands (voice, text, menu, etc.), input in interface fields, etc.”); and
in response to the adjustment operation is a region adjustment operation, performing region adjustment on the interaction navigation panel, the close-range panel, and/or the long-range panel based on the region adjustment operation (KULEWSKI; Col. 19, Lines 8-16; “In block 706, it is checked whether translation between the crop window and the image occurred based on the user input. … the user may have dragged the entire crop window across the image (e.g., using a drag operation) and the image editing interface without resizing the crop window, such that … portion(s) of the crop window are positioned outside the image boundaries. … the user input may have caused the image to be translated relative to the crop window [‘region adjustment operation’].”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method according to claim 1 of GIMPL-BERNSTEIN to include the various teachings of KULEWSKI. The motivation for this modification is to enhance the viewability of the various panels with the virtual space such that the panels may be more viewable to the user by reducing occlusions and arraying the panels in a more user-friendly manner.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN M COFINO whose telephone number is (303) 297-4268. The examiner can normally be reached Monday-Friday 10A-4P MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN M COFINO/ Examiner, Art Unit 2614
/KENT W CHANG/ Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights