DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
2. Claim 1 and 23 are currently amended.
3. Claims 1-29 are pending in the present application.
Regarding Double Patenting
4. The terminal disclaimer filed on 8/5/2026 has been entered and the double patenting rejections in Non-Final Rejection from 4/16/2026 are withdrawn.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
7. Claims 1-8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Evans et al. (US Patent Application Publication No. 2018/0300952 A1) in view of Pollard et al. (US Patent 10,859,831 B1).
8. Regarding Claim 1 (Currently Amended), Evans discloses A computer-implemented method for creating a digital object in an extended reality (XR) environment, comprising: (paragraph [0137] reciting “According to a first aspect, a method, implemented by one or more computing devices, is described for placing a virtual object in a modified-reality environment. …”)
detecting, by an XR hardware device (paragraph [0131] reciting “FIG. 18 more generally shows computing functionality 1802 that can be used to implement any aspect of the mechanisms set forth in the above-described figures. For instance, the type of computing functionality 1802 shown in FIG. 18 can be used to implement the HMD 1702 of FIG. 17, or, more generally, the computing device 1102 of FIG. 11. In all cases, the computing functionality 1802 represents one or more physical and tangible processing mechanisms.” HMD is XR hardware device.) comprising at least one processor, memory, (paragraph [0132] reciting “The computing functionality 1802 can include one or more hardware processor devices 1804, such as one or more central processing units (CPUs), and/or one or more graphics processing units (GPUs), and so on. The computing functionality 1802 can also include any storage resources (also referred to as computer-readable storage media or computer-readable storage medium devices) 1806 for storing any kind of information, such as machine-readable instructions, settings, data, etc. ”) a display, (paragraph [0100] reciting “
One or more output devices 1118 provide a representation of the AR environment 1120. The output devices 1118 can include any combination of display devices, including a liquid crystal display panel, an organic light emitting diode panel (OLED), …”) and one or more input devices, (paragraph [0085] reciting “The computing device 1102
includes a collection of input devices 1104 for interacting with a physical environment
1106, such as the scene depicted in FIGS. 1-9. The input devices 1104 can include, but are not limited to: one or more environment-facing video cameras, an environment-facing depth camera system, a gaze-tracking system, an inertial measurement unit (IMU), one or more microphones, …”) a voice command from a user (paragraph [0088] reciting “… The input processing engine 1108 can also interpret any voice commands issued by the user 102 by analyzing audio input signals supplied by the microphone(s). …”) requesting creation of a digital object for display in the XR environment; (paragraph [0035] reciting “… The user 102 may issue the command “set cube” to instruct the HMD 104 to display a virtual cube. The user 102 may issue the command “set sphere” to place the virtual sphere, and so on. The virtual object that is displayed may have default properties, such as a default size, default orientation, default color, etc. In another example, the HMD 104 can present a drop-down menu in the AR environment through which the user 102 may specify a desired virtual object, etc. Assume, as stated above, that the user selects a virtual cube to be added to the AR environment.”)
detecting, by the XR hardware device, a physical input generated by the user and indicative of a spatial location within the XR environment; (paragraph [0039] reciting “The HMD 104 can use other input modes to identify a point in the AR environment (besides the gaze detection technique, or in addition to the gaze detection technique). For example, in another approach, the HMD 104 can use a body-movement detection engine to determine a direction in which the user 102 is pointing in the AR environment, e.g., using an extended arm and/or finger.” The spatial location is the point being selected as shown in FIG. 1 and FIG. 2 by user gaze or by user finger/hand.) determining, based on the detected physical input, a direction within the XR environment; (paragraph [0039] reciting “The HMD 104 can use other input modes to identify a point in the AR environment (besides the gaze detection technique, or in addition to the gaze detection technique). For example, in another approach, the HMD 104 can use a body-movement detection engine to determine a direction in which the user 102 is pointing in the AR environment, e.g., using an extended arm and/or finger.”) determining three-dimensional spatial coordinates corresponding to the spatial location by determining an intersection point of a ray extending in the determined direction with a virtual boundary in the XR environment (see FIG. 1-2; paragraph [0047] reciting “Next, the user 102 trains his gaze on a desired location on the line 302 at which he wishes to place a virtual object. The gaze detection engine detects the direction of the user's gaze, projects a ray 304 in the identified direction, and determines a point 306 at which the ray intersects the line 302. The point 306 is referred to as a second point herein to help distinguish it from the previously-selected first point 112 on the driveway 110. The user 102 may confirm that the second point 306 is correct by performing an air tap or speaking a command “set point” 308, etc. The line
302 may be regarded as a guide insofar as it assists the user 102 in selecting a y-axis dimension-value.” Line 302 is a ray extending in a direction determined by user gaze (or by hands/fingers). It intersects with a physical boundary of the real world in the AR environment. The 3D spatial coordinate corresponds to the location that the point 112 is located at.) and returning the intersection point as a vector having X, Y, and Z coordinate values; (paragraph [0037] reciting “More specifically, in one non-limiting approach, the HMD 104 uses a gaze detection engine (described in Section B) to determine the direction that the user is looking within the AR environment. The HMD
104 then projects a ray 116 into the AR environment, in the identified direction. The HMD 104 then identifies a point at which the ray 116 intersects a surface within the AR environment. Here, assume that the ray 116 intersects the driveway 110
at the first point 112. In one implementation, the user can move the virtual object 114 to different locations on the driveway 110 by looking at different points on the driveway's surface.”;
paragraph [0074] reciting “… The user 102 may then use any of the strategies described above to select a point 904 on the line 902. For example, the user 102 may train his gaze (corresponding to ray 906) to a desired point along the line 902. The user 102 may confirm his selection of the desired selection by making an air tap or issuing the voice command “set point” 908. In response, the HMD 104 stores value information that defines the final x, y, z placement of the virtual object 114.” The first point 112 once it becomes set point has x,y,z coordinates.)
paragraph [0077] reciting “… HMD 104 projects a line 1006 that extends from the surface of the statue 1004, normal to the point 1002 that has been selected by the user 102.” A normal is a vector point located at x,y,z set point coordinate location.) and displaying, via the XR hardware device, a representation of the digital object at the determined spatial coordinates after both the detected voice command and the detected physical input. (paragraph [0041] reciting “The HMD
104 may operate in conjunction with yet other input modes. However, to simplify explanation, FIGS. 1-9 show the case in which the user 102 chooses a point in the AR environment by training his gaze on that point, and thereafter confirms the selected point using a hand gesture (e.g., an air tap) or a voice command.”;
paragraph [0036] reciting “Next, the user 102 selects a first point on any surface of the AR environment. For example, assume that the user 102 selects a first point
112 on a generally planar surface that corresponds to the driveway 110. The HMD 104
may respond by provisionally placing a virtual cube 114 at the first point 112, e.g., centered at the first point 112 or directly above the first point 112.” The gaze (or finger/pointing) and voice selects the selected point onto which the virtual cube 14 is place.)
While not explicitly disclosed by Evans, Pollard discloses determining three-dimensional spatial coordinates corresponding to the spatial location by determining an intersection point of a ray extending in the determined direction with a virtual boundary in the XR environment (see FIG. 5A and 5B; see col. 11, lines 28-32 reciting “FIGS. 5A and 5B present a perspective view and top view, respectively, of a user interacting with a reproduction 500 of the real-world environment 400 of FIGS. 4A and 4B to produce a virtual boundary or safety boundary, according to some embodiments”;
col. 11, lines 59-62 reciting “FIG. 5B illustrates a top view within the
reproduction 500 of the real-world environment 400 that depicts the intersection point 502, the virtual line 504, and the virtual boundary 506.”)
It would have been obvious to a person of ordinary skills in the art to before the effective filing date of the claimed invention to modify Evans with Pollard so that the virtual boundary is shown in Evans. This is an obviously beneficial modification since the virtual boundary in the AR shows the user where the physical boundaries are and ensure that the user does not inadvertently contact the real world surfaces while viewing the AR world through the head-mounted device (HMD).
9. Regarding Claim 2 (Original), Evans further discloses The method of claim 1, wherein the XR environment comprises one of a virtual reality environment, an augmented reality environment, or a mixed reality environment. (paragraph [0028] reciting “… For instance, the HMD 104 can produce the AR environment using a partially-transparent display device. …”)
10. Regarding Claim 3 (Original), Evans further discloses The method of claim 1, wherein the physical input comprises a pointing gesture. (paragraph [0039] reciting “The HMD 104 can use other input modes to identify a point in the AR environment (besides the gaze detection technique, or in addition to the gaze detection technique). For example, in another approach, the HMD 104 can use a body-movement detection engine to determine a direction in which the user 102 is pointing in the AR environment, e.g., using an extended arm and/or finger.”)
11. Regarding Claim 4 (Original), Evans discloses The method of claim 3, wherein determining the spatial coordinates comprises determining an intersection between a raycast extending from the pointing gesture (paragraph [0037] reciting “More specifically, in one non-limiting approach, the HMD 104 uses a gaze detection engine (described in Section B) to determine the direction that the user is looking within the AR environment. The HMD 104 then projects a ray 116 into the AR environment, in the identified direction. The HMD 104 then identifies a point at which the ray 116
intersects a surface within the AR environment. Here, assume that the ray 116
intersects the driveway 110 at the first point 112. In one implementation, the user can move the virtual object 114 to different locations on the driveway 110 by looking at different points on the driveway's surface.”;
paragraph [0039] reciting " The HMD 104 can use other input modes to identify a point in the AR environment (besides the gaze detection technique, or in addition to the gaze detection technique). For example, in another approach, the HMD 104 can use a body-movement detection engine to determine a direction in which the user 102 is pointing in the AR environment, e.g., using an extended arm and/or finger.”
Line 116 corresponds to a ray line extending from a gaze which can also be pointing finger. )
Pollard further discloses and a virtual object or boundary in the XR environment. (see FIG. 5A and 5B; see col. 11, lines 28-32 reciting “FIGS. 5A and 5B present a perspective view and top view, respectively, of a user interacting with a reproduction 500 of the real-world environment 400 of FIGS. 4A and 4B to produce a virtual boundary or safety boundary, according to some embodiments”;
col. 11, lines 59-62 reciting “FIG. 5B illustrates a top view within the
reproduction 500 of the real-world environment 400 that depicts the intersection point 502, the virtual line 504, and the virtual boundary 506.”)
00. Regarding Claim 5 (Original), Pollard further discloses The method of claim 4, wherein the boundary comprises a virtual representation of a floor or wall. (see FIG. 5A and 5B wherein the virtual boundary is a reproduction of the actual physical environment which comprises a wall and/or floor.)
12. Regarding Claim 6 (Original), Evans further discloses The method of claim 1, wherein the physical input comprises detection of a gaze direction of the user. (paragraph [0037] reciting “More specifically, in one non-limiting approach, the HMD 104 uses a gaze detection engine (described in Section B) to determine the direction that the user is looking within the AR environment. …”)
13. Regarding Claim 7 (Original), Evans further discloses The method of claim 6, wherein determining the spatial coordinates comprises determining an intersection between a direction determined from the detected gaze (paragraph [0037] reciting “… The HMD 104 then identifies a point at which the ray 116 intersects a surface within the AR environment. Here, assume that the ray 116 intersects the driveway 110 at the first point 112. In one implementation, the user can move the virtual object 114 to different locations on the driveway 110 by looking at different points on the driveway's surface.” First point 112 has spatial coordinates and intersects with a surface as a direction of the gaze/finger pointing.)
Pollard further discloses and a virtual object or boundary in the XR environment. (see FIG. 5A and 5B; see col. 11, lines 28-32 reciting “FIGS. 5A and 5B present a perspective view and top view, respectively, of a user interacting with a reproduction 500 of the real-world environment 400 of FIGS. 4A and 4B to produce a virtual boundary or safety boundary, according to some embodiments”;
col. 11, lines 59-62 reciting “FIG. 5B illustrates a top view within the
reproduction 500 of the real-world environment 400 that depicts the intersection point 502, the virtual line 504, and the virtual boundary 506.”)
14. Regarding Claim 8 (Original), Pollard further discloses The method of claim 1, wherein the physical input comprises input from a handheld controller or virtual pointer object. (see FIG. 5A wherein controller 300 is a handheld controller for pointing point 502. It would have been obvious to modify Evans with Pollard’s controller 300 in order to make selecting the first point 112 easier.)
15. Regarding Claim 15 (Original), Evans further discloses The method of claim 1, further comprising generating the representation of the digital object from a stored pre-existing object or from an artificial intelligence image generation process. (paragraph [0035] reciting “With respect to FIG. 1, the user 102 may begin the placement process by specifying a particular virtual object to be added to the AR environment, from among a set of candidate virtual objects. For example, assume that three types of virtual objects are available, corresponding to a cube, a sphere, and a pyramid. The user 102 may issue the command “set cube” to instruct the HMD 104 to display a virtual cube. The user 102 may issue the command “set sphere” to place the virtual sphere, and so on. The virtual object that is displayed may have default properties, such as a default size, default orientation, default color, etc. In another example, the HMD 104 can present a drop-down menu in the AR environment through which the user 102 may specify a desired virtual object, etc. Assume, as stated above, that the user selects a virtual cube to be added to the AR environment.” Virtual objects such as sphere, cube, and pyramid are default pre-existing objects stored in memory for selection.)
16. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Evans in view of Pollard and further in view of Kline et al. (US Patent Application Publication No. 2020/0065889 A1).
17. Regarding Claim 9 (Original), while the combination of Evans and Pollard does not explicitly disclose, Kline discloses The method of claim 1, wherein the physical input comprises input from a glove input device configured to communicate directional information to the XR hardware device. (paragraph [0052] reciting “…
For example, an input from a participant VR device, or component thereof, can be a motion input (e.g., turning a participant's head, detected by a visual VR device, and/or a participant pointing with a hand or finger detected by a somesthetic device such as a VR glove). In another example, an input can be a voice input, such as a voice command spoken by a participant into a VR headset microphone.”
The physical input corresponds to pointing fingers while wearing a VR Glove. The finger pointing is detected by VR device such as HMD/headset.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify the combination of Evans and Pollard with Kline so that user can wear an glove. This allows the user to point his fingers while wearing a glove and have the HMD/headset detect the pointing as disclosed in Evans modified by Kline. This is a beneficial modification as it allows user to point fingers and hands while wearing a protective glove for warmth and comfort, etc.
18. Claims 10 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Evans in view of Pollard and further in view of Anderson et al. (US Patent Application Publication No. 2014/0071069 A1).
19. Regarding Claim 10 (Original), while the combination of Evans and Pollard does not explicitly disclose, Anderson discloses The method of claim 1, further comprising receiving spoken instructions specifying at least one visual attribute of the digital object and displaying the digital object with the specified visual attribute. (paragraph [0065] reciting “As in the examples of FIGS. 1, 2, and 3, and in the virtual binding scenarios, the user may use voice commands to change the nature of the object to be launched (color, category, capabilities) or change the nature of how it is launched or change the view that appears on the displays before and after launching the object. Alternatively, secondary gestures or eye tracking may be used to influence the targeting and the physics of the virtual object. All of these inputs may be accepted and processed before, during, or after launch of the virtual object. These effects may all be received by the Object and Gesture Recognition System and even by cameras and microphones for eye tracking, voice tracking and direction of attention.”)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify the combination of Evans and Pollard with Anderson so user can use voice command to change color or other visual attributes of the digital model in Evans. This is an obvious modification since Evans already discloses changing color of the virtual objects. Modifying Evans with Anderson allows the user to change color of virtual object more easily through voice commands.
20. Regarding Claim 14 (Original), Evans further discloses The method of claim 10, further comprising generating the representation of the digital object from a stored pre-existing object or from an artificial intelligence image generation process. (paragraph [0035] reciting “With respect to FIG. 1, the user 102 may begin the placement process by specifying a particular virtual object to be added to the AR environment, from among a set of candidate virtual objects. For example, assume that three types of virtual objects are available, corresponding to a cube, a sphere, and a pyramid. The user 102 may issue the command “set cube” to instruct the HMD 104 to display a virtual cube. The user 102 may issue the command “set sphere” to place the virtual sphere, and so on. The virtual object that is displayed may have default properties, such as a default size, default orientation, default color, etc. In another example, the HMD 104 can present a drop-down menu in the AR environment through which the user 102 may specify a desired virtual object, etc. Assume, as stated above, that the user selects a virtual cube to be added to the AR environment.” Virtual objects such as sphere, cube, and pyramid are default pre-existing objects stored in memory for selection.)
21. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Evans in view of Pollard in view of Anderson and further in view of Powderly et al. (US Patent Application Publication No. 2018/0307303 A1).
22. Regarding Claim 11 (Original), while the combination of Evans, Pollard and Anderson does not explicitly disclose, Powderly discloses The method of claim 10, wherein the voice command includes at least one of the words "here" and "there." (paragraph [0269] reciting “A user can naturally and quickly manage the placement of a virtual object in the user's environment using multimodal inputs, such as, e.g., a combination of eye gaze, gestures, and voice. For example, a user named Lindsay sits down at the table and gets ready to do some work. She opens her laptop and starts up the desktop-Monitors app on her computer. As the computer is loading, she reaches her hand out above the laptop screen and says “Hey Magic Leap, put Monitors here.” In response to this voice command, the wearable system can automatically launch the monitor screens and place them above her laptop. However, when Lindsay says “Put screens there” while looking over at the wall on the other side of the room, the wearable system can automatically place the screens on the wall across from her. Lindsay could also say “Put halcyon here,” while looking at her desk. The halcyon was initially on her kitchen table, but in response to the voice command, the wearable system can automatically move it to her table surface. As she works, she can use a totem to interact with these objects and adjust their scales to her preference.”)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify the combination of Evans, Pollard, and Anderson with Powderly so user can easily use voice command to place a virtual object. This is an obviously beneficial modification since Evans already discloses selecting a set point upon where to place virtual object. Adding the voice command features allows user to easily place a virtual object atop the selected set point.
23. Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Evans in view of Pollard and further in view of Powderly.
24. Regarding Claim 12 (Original), while the combination of Evans and Pollard does not explicitly disclose, Powderly discloses The method of claim 1, wherein the voice command includes at least one of the words "here" and "there." (paragraph [0269] reciting “A user can naturally and quickly manage the placement of a virtual object in the user's environment using multimodal inputs, such as, e.g., a combination of eye gaze, gestures, and voice. For example, a user named Lindsay sits down at the table and gets ready to do some work. She opens her laptop and starts up the desktop-Monitors app on her computer. As the computer is loading, she reaches her hand out above the laptop screen and says “Hey Magic Leap, put Monitors here.” In response to this voice command, the wearable system can automatically launch the monitor screens and place them above her laptop. However, when Lindsay says “Put screens there” while looking over at the wall on the other side of the room, the wearable system can automatically place the screens on the wall across from her. Lindsay could also say “Put halcyon here,” while looking at her desk. The halcyon was initially on her kitchen table, but in response to the voice command, the wearable system can automatically move it to her table surface. As she works, she can use a totem to interact with these objects and adjust their scales to her preference.”)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify the combination of Evans and Pollard with Powderly so user can easily use voice command to place a virtual object. This is an obviously beneficial modification since Evans already discloses selecting a set point upon where to place virtual object. Adding the voice command features allows user to easily place a virtual object atop the selected set point.
25. Regarding Claim 13 (Original), Evans further discloses The method of claim 12, further comprising generating the representation of the digital object from a stored pre-existing object or from an artificial intelligence image generation process. (paragraph [0035] reciting “With respect to FIG. 1, the user 102 may begin the placement process by specifying a particular virtual object to be added to the AR environment, from among a set of candidate virtual objects. For example, assume that three types of virtual objects are available, corresponding to a cube, a sphere, and a pyramid. The user 102 may issue the command “set cube” to instruct the HMD 104 to display a virtual cube. The user 102 may issue the command “set sphere” to place the virtual sphere, and so on. The virtual object that is displayed may have default properties, such as a default size, default orientation, default color, etc. In another example, the HMD 104 can present a drop-down menu in the AR environment through which the user 102 may specify a desired virtual object, etc. Assume, as stated above, that the user selects a virtual cube to be added to the AR environment.” Virtual objects such as sphere, cube, and pyramid are default pre-existing objects stored in memory for selection.)
26. Claims 16, 24 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Kjallstrom et al. (US Patent Application Publication No. 2016/0196692 A1) in view of Anderson.
27. Regarding Claim 16, Kjallstrom discloses A computer-implemented method Abstract reciting “Systems and methods enabling users to interact with an augmented reality environment are disclosed. …”) for modifying an appearance of an existing digital object (paragraph [0075] reciting “… The builder function may include highlighting an existing cube and changing the material for the building blocks. …”) in an extended reality (XR) environment, (paragraph [0006] reciting “In the first aspect, a machine implemented method for interacting with images in an augmented reality environment is disclosed. …”) comprising: detecting, by the XR hardware device, a physical input generated by the user and indicative of at least one of (i) the existing digital object or (ii) a spatial location or dimension associated with the existing digital object; (paragraph [0048] reciting “As illustrated in FIGS. 1A and 1B, the user 12 may select and “grab” the virtual object 32, and then may reposition the virtual object 32 by moving or “dragging” the virtual object 32 to a location above the real object image 24 in an embodiment. The user 12 may then release the virtual object 32 from the virtual pointer 30, leaving the virtual object 32 to float above the real object image 24. …”
determining a modification parameter based on the voice command and the detected physical input; and (paragraph [0075] reciting “… The builder function may include highlighting an existing cube and changing the material for the building blocks. …” Material is the modification parameters.) displaying, via the XR hardware device, the existing digital object with the modified visual attribute according to the modification parameter. (see FIG.10-12; paragraph [0075] reciting “… In one or more embodiments, virtual control button 1032 (“create”) creates a new virtual object at the location of the virtual cross-hair target 1030, virtual control button 1034 (“delete”) deletes the virtual object at the location of the virtual cross-hair target 1030, and virtual control button 1036 (“material”) changes the composition or material of the virtual object.”)
While Kjallstrom does not explicitly disclose, Anderson discloses detecting, by an XR hardware device, a voice command from a user requesting modification of a visual attribute of the existing digital object; (paragraph [0065] reciting “As in the examples of FIGS. 1, 2, and 3, and in the virtual binding scenarios, the user may use voice commands to change the nature of the object to be launched (color, category, capabilities) or change the nature of how it is launched or change the view that appears on the displays before and after launching the object.” A request for change is a modification and the color is the visual attribute of the existing virtual object that is changed.)
determining a modification parameter based on the voice command and the detected physical input; and (paragraph [0065] reciting “As in the examples of FIGS. 1, 2, and 3, and in the virtual binding scenarios, the user may use voice commands to change the nature of the object to be launched (color, category, capabilities) or change the nature of how it is launched or change the view that appears on the displays before and after launching the object.” )
displaying, via the XR hardware device, the existing digital object with the modified visual attribute according to the modification parameter. (paragraph [0065] reciting “As in the examples of FIGS. 1, 2, and 3, and in the virtual binding scenarios, the user may use voice commands to change the nature of the object to be launched (color, category, capabilities) or change the nature of how it is launched or change the view that appears on the displays before and after launching the object.” )
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Kjallstrom with Anderson so that the voice command can change color (material) of the virtual object. This is an obvious modification since Kjallstrom already discloses changing the material composition of the virtual object and color would be such a material composition that can be changed easily with voice command as disclosed in Anderson.
28. Regarding Claim 24, Kjallstrom further discloses The method of claim 16, wherein the XR hardware device repositions the existing digital object relative to another object identified by the physical input. (paragraph [0061] reciting “FIG. 1H is a front, perspective view of a user 12 selecting and disassembling a virtual object
62 from a group of virtual objects 60. A group of objects 60 comprises virtual objects 62, 64, 66, and 68. In one or more embodiments, the group of objects 60
represents an assembled part or component where the virtual objects 62, 64, 66, and
68 are physically coupled. The user 12 points to, selects, and disassembles virtual object 62 from the group 60 of virtual objects. As shown in FIG. 1I, the virtual pointer
30 secures or “grabs” the virtual object 62, and the user 12 may reposition virtual object 62 to a new location.”
Virtual object 62 is repositioned relative to other virtual objects.)
29. Regarding Claim 29, Kjallstrom further discloses The method of claim 16, further comprising generating the existing digital object with the modified visual attribute from a stored pre-existing object or from an artificial intelligence image generation process. (paragraph [0068] reciting “… Selecting a region of an augmented reality environment may refer to selecting a point, area, or volume within the augmented reality environment 10 such that the user 12 may create or paste a new object at the location defined by the region, or may otherwise manipulate the region of the augmented reality environment 10. …” Pasting an object means it is created by copying or generating a copy of a pre-existing virtual object.)
30. Claims 17 is rejected under 35 U.S.C. 103 as being unpatentable over Kjallstrom in view of Anderson further in view of Zhou et al. (US Patent Application Publication No. 2018/0267688 A1).
31. Regarding Claim 17, while the combination of Kjallstrom and Anderson does not explicitly disclose, Zhou discloses The method of claim 16, wherein the physical input comprises a pointing gesture directed to a portion of the existing digital object. see FIG. 2B; paragraph [0037] reciting “Further, when the first parameter of the operating entity is the posture of the operating entity, to specifically determine whether the first parameter of the operating entity is a fist or index finger extension, etc., a device having an image recognition function may be applied to perform analysis and determination. …”;
paragraph [0061] in view of “In the example shown in FIG. 2B, the operating entity 221 is located outside of the display space of the virtual object. That is, the first parameter of the operating entity 221 satisfies the second condition. Thus, the pointer 222 is displayed, and control of the virtual object is performed through the pointer 222. The location of the pointer 222 may be obtained by mapping the location of the operating entity 221 to the display space of the virtual object based on the location mapping relationship between the space region of the operating entity 221 and the display space of the virtual object. The space region of the operating entity 221 may refer to the space outside of the display space of the virtual object and within the detection range of the detection device such as the depth camera. An example of the location of the pointer 222 may be found in FIG. 2.”
The index finger is considered an operating entity and image analysis is used to determined where index finger is pointing and the virtual cursor is aligned with the direction the index finger is pointing towards.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Kjallstrom and Anderson with Zhou so that instead of using a cursor the user can use the fingers to control positioning of the virtual pointer. This is an obviously beneficial modification since using fingers to control the virtual pointer is intuitive to the user and facilitates user selection of region in the AR environment to paste or create a virtual object.
32. Claims 18 is rejected under 35 U.S.C. 103 as being unpatentable over Kjallstrom in view of Anderson and further in view of Salter et al. (US Patent Application Publication No. 2014/0152558 A1).
33. Regarding Claim 18, while the combination of Kjallstrom and Anderson does not explicitly disclose, Salter discloses The method of claim 16, wherein the physical input comprises detection of a gaze direction used to select the existing digital object. (paragraph [0003] reciting “Technology is described for facilitating control of an augmented reality environment associated with a head-mounted display device (HMD). In some embodiments, a virtual pointer may be displayed to an end user of the HMD and controlled by the end user using motion and/or orientation information associated with a secondary device (e.g., a mobile phone). Using the virtual pointer, the end user may select and manipulate virtual objects within the augmented reality environment, select real-world objects within the augmented reality environment, and/or control a graphical user interface of the HMD. In some cases, the initial position of the virtual pointer within the augmented reality environment may be determined based on a particular direction in which the end user is gazing and/or a particular object at which the end user is currently focusing on or has recently focused on.” A gaze direction results in a cursors positioning in the virtual world.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Kjallstrom and Anderson with Salter so gaze is used to project the virtual point in Kjallstrom. This is a beneficial modification since gaze is intuitive and facilitates the selection of a virtual object.
34. Claims 19 is rejected under 35 U.S.C. 103 as being unpatentable over Kjallstrom in view of Anderson and further in view of Paul Lacey (US Patent Application Publication No. 2021/0263593 A1).
35. Regarding Claim 19, The method of claim 16, wherein the modification comprises resizing the existing digital object based on a detected distance between two user hands. (paragraph [0106] reciting “Various types of bimanual interactions may fall into one of three categories. The first category is independent bimanual interaction, in which each hand interacts with virtual objects independently (e.g., a user is typing on a virtual keyboard and each hand configuration is independent of the other). The second category is cooperative bimanual interaction, in which both hands cooperatively interact with virtual objects (e.g., resizing, rotating, and/or translating a virtual cube by pinching opposite corners with both hands). The third category is managed bimanual interaction, in which one hand manages how the other hand is interpreted (e.g., right hand is cursor while left hand is qualifier that switches cursor between pen and eraser).”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Kjallstrom and Anderson with Lacey so the user can resize a virtual object with two hands. This allows for further manipulation of virtual objects in an augmented reality which is a feature of Kjallstrom. Therefore, this modification is obviously beneficial.
36. Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Kjallstrom in view of Anderson and further in view of Sandeep Verma (US Patent Application Publication No. 2023/0089622 A1).
37. Regarding Claim 20, while the combination of Kjallstrom and Anderson does not explicitly disclose, Verma discloses The method of claim 16, wherein the modification comprises resizing based on a detected distance between two fingers of one hand. (paragraph [0048] reciting “As another example, providing access to the virtual environment 400 may allow the user to use hand gestures to rescale virtual objects 402 within the virtual environment 400. In this example, the augmented reality device 104 may detect hand gestures performed by the user that identifies a virtual object 402 and a scale or size change for the virtual object 402. For instance, the user may pinch two fingers together to indicate a decrease in the size of the virtual object 402 or pull two fingers apart to indicate an increase in the size of the virtual object 402. The augmented reality device 104 may then rescale or resize the identified virtual object 402 based on the detected hand gestures. In other examples, the augmented reality device 104 may perform any other suitable type of action on virtual objects 402 within the virtual environment 400 based on detected gestures from the user.” Finger pinching occurs on one hand.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Kjallstrom and Anderson with Verma so the user can resize a virtual object with finger pinching on one hand. This allows for further manipulation of virtual objects in an augmented reality which is a feature of Kjallstrom. Therefore, this modification is obviously beneficial.
38. Claims 25 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Kjallstrom in view of Anderson and further in view of Sol E. Choi (US Patent Application Publication No. 2023/0215101 A1).
39. Regarding Claim 25, while the combination of Kjallstrom and Anderson does not explicitly disclose, Choi discloses The method of claim 16, wherein the XR hardware device spawns the modified digital object at a default location relative to a field of view of the user when no valid spatial location is detected. (paragraph [0092] reciting “The user may set, as a default position of the 3D object 632, a position of the 3D object 632 that is provided in front of the avatar 411 or behind the avatar 411
in the virtual space 410 through the position setting screen 740.” The default position to place created new virtual object can be set by user.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Kjallstrom and Anderson with Choi so user can set default region to place created virtual objects. This is an obviously beneficial modification since virtual objects need at least a default location to be rendered when displayed.
40. Regarding Claim 28, Kjallstrom further discloses The method of claim 25, further comprising generating the existing digital object with the modified visual attribute from a stored pre-existing object or from an artificial intelligence image generation process. (paragraph [0068] reciting “… Selecting a region of an augmented reality environment may refer to selecting a point, area, or volume within the augmented reality environment 10 such that the user 12 may create or paste a new object at the location defined by the region, or may otherwise manipulate the region of the augmented reality environment 10. …” Pasting an object means it is created by copying or generating a copy of a pre-existing virtual object.)
41. Claims 26-27 are rejected under 35 U.S.C. 103 as being unpatentable over Kjallstrom in view of Anderson and further in view of Powderly.
42. Regarding Claim 26, while the combination of Kjallstrom and Anderson does not explicitly disclose, Powderly discloses The method of claim 16, wherein the voice command includes a word indicating location in the XR environment. (paragraph [0269] reciting “A user can naturally and quickly manage the placement of a virtual object in the user's environment using multimodal inputs, such as, e.g., a combination of eye gaze, gestures, and voice. For example, a user named Lindsay sits down at the table and gets ready to do some work. She opens her laptop and starts up the desktop-Monitors app on her computer. As the computer is loading, she reaches her hand out above the laptop screen and says “Hey Magic Leap, put Monitors here.” In response to this voice command, the wearable system can automatically launch the monitor screens and place them above her laptop. However, when Lindsay says “Put screens there” while looking over at the wall on the other side of the room, the wearable system can automatically place the screens on the wall across from her. Lindsay could also say “Put halcyon here,” while looking at her desk. The halcyon was initially on her kitchen table, but in response to the voice command, the wearable system can automatically move it to her table surface. As she works, she can use a totem to interact with these objects and adjust their scales to her preference.”)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Kjallstrom and Anderson with Powderly so that the keywords like “here” and “there” are recognized as keywords indicating the creation and rendering of a virtual object. This is an obvious modification since Kjallstrom already discloses using voice command to create an object and Powderly allows common words to be recognized as keywords to create and rendering a virtual object at a particularly selected region of the virtual environment.
43. Regarding Claim 27, Kjallstrom further discloses The method of claim 26, further comprising generating the existing digital object with the modified visual attribute from a stored pre-existing object or from an artificial intelligence image generation process. (paragraph [0068] reciting “… Selecting a region of an augmented reality environment may refer to selecting a point, area, or volume within the augmented reality environment 10 such that the user 12 may create or paste a new object at the location defined by the region, or may otherwise manipulate the region of the augmented reality environment 10. …” Pasting an object means it is created by copying or generating a copy of a pre-existing virtual object.)
Allowable Subject Matter
44. Claims 21-23 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
45. The following is a statement of reasons for the indication of allowable subject matter: Claim 21 recites the limitation wherein the modification comprises changing a color of the existing digital object based on a spoken instruction referencing another object which is neither disclosed nor suggested by the cited references, either singly or in combination. Closest art of record is Evans which discloses changing color of a virtual object but Evans fails to disclose or suggest limitation. Anderson discloses using voice command to change color or other visible qualities of a virtual object, but Anderson fails to disclose this limitation.
46. Claim 22 recites the limitation The method of claim 16, further comprising displaying a measurement selection interface including measurement markings for selection of a measurable dimension which is neither disclosed nor suggested by the cited references, either singly or in combination. Closest art of record is Evans which discloses changing dimensions of a virtual object but Evans fails to disclose or suggest limitation. Anderson discloses using voice command to change color or other visible qualities of a virtual object, but Anderson fails to disclose this limitation.
47. Claim 23 recites the limitation The method of claim 16, wherein the measurable dimension comprises volume, and the modified visual attribute comprises liquid volume represented within the digital object which is neither disclosed nor suggested by the cited references, either singly or in combination. Closest art of record is Evans which discloses changing visual attributes of a virtual object but Evans fails to disclose or suggest limitation. Anderson discloses using voice command to change color or other visible qualities of a virtual object, but Anderson fails to disclose this limitation.
Response to Arguments
48 Applicant’s arguments, see Remarks, filed on 7/16/2026, with respect to the rejection(s) of claim(s) 1, 2, 8, and 15 under Kjallstrom and Faulkner have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Evans modified by Pollard. Previously unused prior art has been cited for the rejection of claims 1-15 due to substantial change in scope of claim 1. The crucial element a spatial location has been further defined in the claims, thereby allowing new arts to be used to reject amended claim 1.
49. Regarding the rejection of claim 1, Evans discloses most of the limitations while Pollard provides the virtual point intersecting the virtual boundary of the extended reality scene. Please review the above rejection of claims 1-15 for further understanding.
50. On page 10 of the Remarks, Applicants argue that 13-15 reliance of Kjallstrom to disclose artificial intelligence image generation process is invalid. However, 13-15 now relies on Evans which discloses pre-existing virtual objects, such as a sphere, cube, or pyramid.
51. On page 10 of the Remarks, Applicants argue that 27-29 reliance on Kjallstrom to disclose artificial intelligence image generation process is invalid. Examiner concedes that applicants are correct in that no AI is used. However, claims 27-29 recites the options of from a stored pre-existing object or from an artificial intelligence image generation process. Kjallstrom at paragraph [0067] recites in part “… Selecting a region of an augmented reality environment may refer to selecting a point, area, or volume within the augmented reality environment 10 such that the user 12 may create or paste a new object at the location defined by the region, or may otherwise manipulate the region of the augmented reality environment 10. …” Pasting a new object at a selected location means generating a new object based on preexisting object.
52. Applicants argue that Kjallstrom and Anderson together fails to disclose “determining a modification parameter based on the voice command and the detected physical input”. Examiner disagrees. The modification parameter in this instance is the color of a virtual object. Anderson clearly discloses in paragraph [0103] recites in part that “… the user could use voice commands to change the nature of the object to be launched (color, category, capabilities) or to change the nature of how it is launched, or to change the view that appears on the displays as the user does it.” This means there must be at least 2 color or else there cannot be a change in color. Therefore, 1 of at least 2 colors is chosen based on voice command. Again, the color is the modification parameter. Kjallstrom at paragraph [0048] discloses selecting (grabbing) a virtual object to have actions performed on that virtual object. The grabbing is the physical component. Therefore, both physical and voice command are used to modify the color parameter (selecting actual color) of a virtual object. The list of colors that object can be corresponds to the visual attribute and the changing of the color of the selected virtual object to a specific color (modification parameter) is achieved by grabbing that virtual object and changing its color with voice command. Therefore, Kjallstrom and Anderson discloses all the limitations of claim 16.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
CONTACT
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK S CHEN whose telephone number is (571)270-7993. The examiner can normally be reached Mon - Fri 8-11:30 and 1:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 5712727794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK S CHEN/Primary Examiner, Art Unit 2611
Prosecution Insights