Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Applicant’s Arguments
Applicant’s arguments filed 08/24/2026 have been fully considered, but they are not deemed to be persuasive. Applicant argues that Reitan does not teach "the first annotation corresponds to the first position,” and the amended "wherein the first annotation is different from the displayed first representation." However, Reitan’s virtual object 1301 correspondence to one of the available positions 1302, 1303, and 1319 (e.g., [1106] - Processor 1317 may determine to place an object 1301 at location 1319. Once a determination to place an object 1301 has been made, a processor 1317 may place object 1301 at location 1302, 1303, and/or 1319) in which the virtual object can be “annotation” such as a sign, a billboard, a dialogue, or a markup object (i.e., highlighted), … (e.g., [1100] - Object 1301 may be any type of object (or portion thereof). For example, object 1301 may include, but is not limited: food, drinks, furniture, clothing, a logo, a sign, a vehicle, a billboard, a building, athletic equipment, an electronic device, a painting, a person, an animal, scenery, etc; [1120] - In one embodiment, I/O device 1316 may receive dialogue from a plurality of users 1312, 1314, and 1315. Dialogue may comprise any speech, for example a discussion about a piece of clothing a woman is wearing. In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue; [1117] - For example, in some embodiments object 1301 is highlighted (e.g., made prominent or emphasized). Highlighting may include, but is not limited to: making an object 1301 shake or move, adding a shimmer or other special effect to an object 1301, adding a glow to an object 1301, producing a sound, making an object 1301 change color, etc. This list is not meant to be exhaustive) teaches the claimed "the first annotation corresponds to the first position”; furthermore, Reitan also suggests the amended “wherein the first annotation is different from the displayed first representation” in which the virtual object 1301 with additional content 1305 to show the annotation (e.g., the dialogue as a speech) is different from the displayed representation of the dialogue (e.g., additional information to indicate which of the viewers 1312, 1314, and 1315 speaks) (e.g., [1120] - In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue. For example, a processor 1317 may be operable to distinguish between different viewers 1312, 1314, and 1315). Furthermore, the new reference Wang teaches “the first annotation is different from the displayed first representation” (e.g., [0091] - Users may then collaborate by referring to aspects of 3D content, moving around to view different angles, etc. In some embodiments, annotations to virtual content can be made available to collaborating users in real time. For example, a first user may add virtual markup and/or comments to virtual content, and a second user may see the virtual markup and/or comments as the first user creates them; [0093] - In some embodiments, a virtual comment can include location indicator 814 and/or comment bubble 816. In some embodiments, comment bubble 816 can include a visual indicator corresponding to location indicator 814 (e.g., comment bubble 816 and location indicator 814 can share a number). In some embodiments, location indicator 814 may be presented to user 804 in the same position (e.g., relative to the real world, and/or relative to other virtual content) as it is presented to user 802. In some embodiments, comment bubble 816 can be presented in different positions for different users. For example, comment bubble 816 may be presented to user 802 as facing user 802, and comment bubble 816 may be presented to user 804 as facing user 804. In some embodiments, comment bubble 816 may be configured to continually face a user as a user looks in different locations). Accordingly, the claimed invention as represented in the claims does not represent a patentable distinction over the art of record.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over REITAN (2013/0249947) in view of ESCUDERO et al (The making of Designing Holograms), and further in view of WANG et al (WO 2021163373).
As per claim 1, Reitan teaches the claimed "method" comprising: "at a computer system in communication with one or more input devices and a display: while the computer system is participating in a communication session that includes one or more participants" (Reitan, [1099] - FIG. 13A shows an example interactive device 1310. Viewers 1312, 1314 and 1315 are able to watch content on the display 1311 of interactive device 1310. In various embodiments, content includes still video, still images, and/or audio. The content in FIG. 13A shows an office where one person is sitting at a desk and another person is sitting in a chair), and while displaying a virtual scene from a viewpoint of a user of the computer system within a three-dimensional environment of the user: "obtaining, via the one or more input devices, first information including a request to display a first annotation at a position within the three-dimensional environment wherein correspondence of the first annotation to a respective position within the three-dimensional environment is determined based on a context of the user's interaction with the three-dimensional environment" (Reitan, [1106] - For example, processor 1317 may determine that the scene shown in FIG. 13A has available locations 1302, 1303, and 1319 to place an object 1301. Processor 1317 may also determine that this particular scene is shown for a particular amount of time (e.g., the conversation in the scene lasts two minutes, and begins at a particular time in the show). Processor 1317 may determine to place an object 1301 at location 1319. Once a determination to place an object 1301 has been made, a processor 1317 may object 1301 at location 1302, 1303, and/or 1319; [1100] - For example, object 1301 in FIG. 13A is a soda can. When a viewer 1312 sees the soda can he may be more likely to buy that type of soda the next time he buys soda. Object 1301 may be any type of object (or portion thereof). For example, object 1301 may include, but is not limited: food, drinks, furniture, clothing, a logo, a sign, a vehicle, a billboard, a building, athletic equipment, an electronic device, a painting, a person, an animal, scenery, etc. In various embodiments object 1301 is three dimensional (3D). In some embodiments object 1301 is two dimensional (2D). Also, an object 1301 may be opaque, transparent, or translucent; [1117] - For example, in some embodiments object 1301 is highlighted (e.g., made prominent or emphasized); [1120] - In one embodiment, I/O device 1316 may receive dialogue from a plurality of users 1312, 1314, and 1315. Dialogue may comprise any speech); and "in response to obtaining the first information: in accordance with a determination that the first annotation corresponds to a first position in the virtual scene, displaying a first representation of the first annotation at the first position" (Reitan, [1122] - In operation 1321, in one embodiment, a processor 1317 determines available locations 1302, 1303, and 1319 and times within a piece of content to place an object 1301. In an embodiment processor 1317 determines when and/or where to place an object 1301 based at least in part on an available location 1302, 1303 and 1319 and/or time within a piece of content). It is noted that Reitan does not explicitly teach "in accordance with a determination that the first annotation corresponds to a second position in the virtual scene, displaying the first representation of the first annotation at the second position"; however, Reitan's available locations 1302, 1303, and/or 1309 (e.g., [1106], [1102]) suggests that the object 1301 can be placed at a first position (e.g., location 1302) and/or a second location (e.g., location 1303 or 1319) (see also Escudero, page 8, Spatial Awareness (see the video clip cited as
https://leern.miorosoftcom/shows/Docs-Mxed-ReelitV/Mcrosofts-Designing-Hiolograms- Spatial-Awareness-Chapter/player - at 01:36-06:25, the placed location can be adjusted for the scene's consistency; furthermore, the object can be placed at multiple selected locations).
Applicant’s arguments filed 08/24/2026 have been fully considered, but they are not deemed to be persuasive. Applicant argues that Reitan does not teach "the first annotation corresponds to the first position,” and the amended "wherein the first annotation is different from the displayed first representation." However, Reitan’s virtual object 1301 correspondence to one of the available positions 1302, 1303, and 1319 (e.g., [1106] - Processor 1317 may determine to place an object 1301 at location 1319. Once a determination to place an object 1301 has been made, a processor 1317 may place object 1301 at location 1302, 1303, and/or 1319) in which the virtual object can be “annotation” such as a sign, a billboard, a dialogue, or a markup object (i.e., highlighted), … (e.g., [1100] - Object 1301 may be any type of object (or portion thereof). For example, object 1301 may include, but is not limited: food, drinks, furniture, clothing, a logo, a sign, a vehicle, a billboard, a building, athletic equipment, an electronic device, a painting, a person, an animal, scenery, etc; [1120] - In one embodiment, I/O device 1316 may receive dialogue from a plurality of users 1312, 1314, and 1315. Dialogue may comprise any speech, for example a discussion about a piece of clothing a woman is wearing. In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue; [1117] - For example, in some embodiments object 1301 is highlighted (e.g., made prominent or emphasized). Highlighting may include, but is not limited to: making an object 1301 shake or move, adding a shimmer or other special effect to an object 1301, adding a glow to an object 1301, producing a sound, making an object 1301 change color, etc. This list is not meant to be exhaustive) teaches the claimed "the first annotation corresponds to the first position”; furthermore, Reitan also suggests the amended “wherein the first annotation is different from the displayed first representation” in which the virtual object 1301 with additional content 1305 to show the annotation (e.g., the dialogue as a speech) is different from the displayed representation of the dialogue (e.g., additional information to indicate which of the viewers 1312, 1314, and 1315 speaks) (e.g., [1120] - In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue. For example, a processor 1317 may be operable to distinguish between different viewers 1312, 1314, and 1315). Furthermore, the new reference Wang teaches “the first annotation is different from the displayed first representation” (e.g., [0091] - Users may then collaborate by referring to aspects of 3D content, moving around to view different angles, etc. In some embodiments, annotations to virtual content can be made available to collaborating users in real time. For example, a first user may add virtual markup and/or comments to virtual content, and a second user may see the virtual markup and/or comments as the first user creates them; [0093] - In some embodiments, a virtual comment can include location indicator 814 and/or comment bubble 816. In some embodiments, comment bubble 816 can include a visual indicator corresponding to location indicator 814 (e.g., comment bubble 816 and location indicator 814 can share a number). In some embodiments, location indicator 814 may be presented to user 804 in the same position (e.g., relative to the real world, and/or relative to other virtual content) as it is presented to user 802. In some embodiments, comment bubble 816 can be presented in different positions for different users. For example, comment bubble 816 may be presented to user 802 as facing user 802, and comment bubble 816 may be presented to user 804 as facing user 804. In some embodiments, comment bubble 816 may be configured to continually face a user as a user looks in different locations).
Thus, it would have been obvious, in view of Escudero and Wang, to configure Reitan's method as claimed by placing an object in form of an annotation at multiple locations on the display scene. The motivation is to enhance the apparent realistic of objects on the augmented reality space.
Claim 2 adds into claim 1 "wherein the first representation includes a positional marker including a graphical representation indicating one or more portions of the virtual scene" (Reitan, Figure 13A - locations 1302, 1303, 1319).
Claim 3 adds into claim 1 "wherein the first representation includes a simulated marking having one or more visual characteristics based on a simulated marking implement selected while the first information is obtained" (Reitan, [1106] - For example, processor 1317 may determine that the scene shown in FIG. 13A has available locations 1302, 1303, and 1319 to place an object 1301; [1117] - an object 1301 is rendered such that it appears to be part of the original content (e.g., the object 1301 looks like it belongs in the scene). In some embodiments, an object 1301 or content is rendered such that an indication is made to viewer 1312 that viewer 1312 can interact with object 1301. For example, in some embodiments object 1301 is highlighted (e.g., made prominent or emphasized). Highlighting may include, but is not limited to: making an object 1301 shake or move, adding a shimmer or other special effect to an object 1301, adding a glow to an object 1301, producing a sound, making an object 1301 change color, etc.; [1106] - In some embodiments, rendering can adjust the focal length, position, and/or orientation of an object 1301).
Claim 4 adds into claim 1 "wherein the context of the user's interaction with the three-dimensional environment is based on a target of attention of the user, the method further comprising: before displaying the first representation of the first annotation, displaying, via the display, a visual indication of the target of the attention of the user" (Escudero, page 8, Head Tracking and Eye Tracking (see the video clip cited as :/ea.microsoft.com/shows/Docs-Mked-Realty/Micrsofts-Designing-Holograms Head-Tracking-and-Eye-Tracking-Chapter/player) - at 00:14-10:48, the eye gaze tracking performs the target attention of the user and rendering a visual indication of the target based on the context of the user attention on the scene. E.g., on the sofa, top of table, ...). Thus, it would have been obvious, in view of Escudero and Wang, to configure Reitan's method as claimed by determining the target attention, or the location to place the object, based on the context of user attention on the scene. The motivation is to enhance the apparent realistic of objects on the augmented reality space.
Claim 5 adds into claim 4 "wherein the target of the attention of the user is based on a position of a gaze of the user relative to the three-dimensional environment" (Escudero, page 8, Head Tracking and Eye Tracking (see the video clip cited as https://learn.nicrosoftcom/shows/Docs-Miked-Realty/Mficrosofts-Designing-olograne- Head-Tracking-and-Eye-Tracking-Chapter/player) - at 00:14-10:48, the eye gaze tracking performs the target attention of the user and rendering a visual indication of the target at the position of the user's gaze). Thus, it would have been obvious, in view of Escudero and Wang, to configure Reitan's method as claimed by determining the target attention, or the location to place the object, based on a position of a gaze of the user. The motivation is to enhance the apparent realistic of objects on the augmented reality space.
Claim 6 adds into claim 4 "wherein the target of the attention of the user is based on a spatial relationship between one or more portions of a body of the user, the viewpoint of the user, and the three-dimensional environment" (Escudero, page 8, Spatial Awareness (see the video clip cited as https://learm.microsoft.com/shows/Docs- Mixed-Reality/Microsofts-Designing-Holograms-SpatialAwareness-Chapter/player) -al 01:36-06:25, the placed location can be adjusted for the scene's consistency; furthermore, the object can be adjusted and/or placed at multiple selected locations which are defined from the user's body relative to the three-dimensional environment). Thus, it would have been obvious, in view of Escudero and Wang, to configure Reitan's method as claimed by determining the target attention, or the location to place the object, based on a spatial relationship between one or more portions of a body of the user, the viewpoint of the user, and the three-dimensional environment. The motivation is to enhance the apparent realistic of objects on the augmented reality space.
Claim 7 adds into claim 1 "wherein the first information is provided by a participant of the communication session different from the user of the computer system" (Reitan, [1119] - an object 1301 is an interactive gateway to advertisements 1306. In other words, in some embodiments, viewer 1312 receives additional content 1305 by interacting with object 1301; [1120] - In one embodiment, I/O device 1316 may receive dialogue from a plurality of users 1312, 1314, and 1315. Dialogue may comprise any speech, for example a discussion about a piece of clothing a woman is wearing. In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue), and "the first annotation has one or more visual characteristics that visually indicate that the participant provided the first information" (Reitan, [1120] - In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue. For example, a processor 1317 may be operable to distinguish between different viewers 1312, 1314, and 1315).
Claim 8 adds into claim 1 "while displaying the first representation of the first annotation, detecting, via the one or more input devices, an input directed to the first representation of the first annotation; and in response to detecting the input directed to the first representation, presenting contextual information included in the first information" (Reitan, [1116] - In some embodiments, the viewer 1312 can move and/or manipulate an object 1301 using I/O device 1316. For example, viewer 1312 may click on an object 1301 by making gestures (e.g., pointing at an object and pretending to shoot it) recognized by a motion sensor; [0655] - Various embodiments include specific state machine designs comprising the following states: DENIAL, ANGER, BARGAINING, DEPRESSION, ACCEPTANCE wherein next-state transitions are governed by state transition logic based on contextual parsing of dialogue and behavior such that the states represent meaning assigned to individual and/or group expression providing context to parsing of dialogue and other interactions).
Claim 9 adds into claim 8 "wherein the presenting of the contextual information includes presenting audio associated with the first annotation recorded by a participant of the communication session" (Reitan, [1116] - Other examples of interacting with object 1301 include, but are not limited to: making a throwing or kicking motion, speaking in a microphone, talking with other viewers 1314 and 1315, clicking on a mobile I/O device 1308, having a dialogue with other users 1314 and 1315, clapping, etc. In one embodiment, clicking on an object 1301 will provide a viewer 1312 with additional content 1305. In an embodiment a processor 1317 is operable to capture voices of a plurality of viewers 1312, 1314, and 1315).
Claim 10 adds into claim 1 "while the first representation of the first annotation is included within the three-dimensional environment, obtaining second information, different from the first information, requesting display of a user interface directed to displaying a plurality of representations of a plurality of annotations including the first annotation" (Reitan, [1116] - In some embodiments, the viewer 1312 can move and/or manipulate an object 1301 using I/O device 1316; [1110] - In an embodiment interactive device 1310 is operable to provide a viewer 1312 with additional content 1305 comprising interactive advertising)); and "in response to obtaining the second information, displaying, via the display, at least a portion of the plurality of representations of the plurality of annotations, including a second representation of the first annotation, different from the first representation of the first annotation" (Reitan, [1117] - an object 1301 is rendered such that it appears to be part of the original content (e.g., the object 1301 looks like it belongs in the scene). In some embodiments, an object 1301 or content is rendered such that an indication is made to viewer 1312 that viewer 1312 can interact with object 1301. For example, in some embodiments object 1301 is highlighted (e.g., made prominent or emphasized). Highlighting may include, but is not limited to: making an object 1301 shake or move, adding a shimmer or other special effect to an object 1301, adding a glow to an object 1301, producing a sound, making an object 1301 change color, etc.; [1106] - In some embodiments, rendering can adjust the focal length, position, and/or orientation of an object 1301).
Claim 11 adds into claim 1 "wherein the first information includes an air gesture performed by one or more portions of a body of the user" (Reitan, [0071] - It is noted that there are a variety of methods for a user to control the position, or orientation, of virtual viewport 137 in accordance with various embodiments. For example, a keyboard, joystick, touchpad, voice control, a virtual control panel, camera-based gesture recognition, etc; [1116] - In some embodiments, the viewer 1312 can move and/or manipulate an object 1301 using I/O device 1316. For example, viewer 1312 may click on an object 1301 by making gestures (e.g., pointing at an object and pretending to shoot it) recognized by a motion sensor).
Claim 12 adds into claim 1 "while the computer system is participating in the communication session that includes the one or more participants, and while displaying the virtual scene from the viewpoint of the user of the computer system, obtaining, via the one or more input devices, second information including a request to include a second annotation within the three-dimensional environment" (Reitan, [1120] - In one embodiment, I/O device 1316 may receive dialogue from a plurality of users 1312, 1314, and 1315. Dialogue may comprise any speech, for example a discussion about a piece of clothing a woman is wearing. In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue. For example, a processor 1317 may be operable to distinguish between different viewers 1312, 1314, and 1315; [0962] - Users 1112, 1113 and 1114 are shown watching the display 1111 of interactive device 1110. The display 1111 shows an example presentation layer (e.g., a layer displaying content 1105, interactive elements 1106, scroll bar 1107, and avatar 1101) Interactive device may include, but is not limited to: computers, televisions, radios, interactive televisions, video game consoles, mobile devices, smart phones, smart televisions, automobile consoles, windshields, laptops, personal digital assistants, tablet computers, etc; [1106] - For example, processor 1317 may determine that the scene shown in FIG. 13A has available locations 1302, 1303, and 1319 to place an object 1301. Processor 1317 may also determine that this particular scene is shown for a particular amount of time (e.g., the conversation in the scene lasts two minutes, and begins at a particular time in the show). Processor 1317 may determine to place an object 1301 at location 1319. Once a determination to place an object 1301 has been made, a processor 1317 may place object 1301 at location 1302, 1303, and/or 1319; [1100] - For example, object 1301 in FIG. 13A is a soda can. When a viewer 1312 sees the soda can he may be more likely to buy that type of soda the next time he buys soda. Object 1301 may be any type of object (or portion thereof). For example, object 1301 may include, but is not limited: food, drinks, furniture, clothing, a logo, a sign, a vehicle, a billboard, a building, athletic equipment, an electronic device, a painting, a person, an animal, scenery, etc. In various embodiments object 1301 is three dimensional (3D). In some embodiments object 1301 is two dimensional (2D). Also, an object 1301 may be opaque, transparent, or translucent) (Noted: each of the users 1112, 1113, and 1114 can access to the display 1111 through the interactive 1106 through its own computer for, e.g., placing objects on selected locations); and in response to obtaining the second information, presenting feedback indicating a spatial relationship between a position of the second annotation within the three-dimensional environment and the viewpoint of the user" (Escudero, page 8, Spatial Awareness (see the video clip cited as https://lean.microsoftcom/shows/Docs-Mxec-Reality/Micrcsofts-Desiging-Holograms- Spatial-Awareness-Chapter/ptayer) - at 01:36-06:25, the placed location can be adjusted for the scene's consistency; furthermore, the object can be adjusted and/or placed at multiple selected locations which are defined from the user's body relative to the three-dimensional environment). Thus, it would have been obvious, in view of Escudero and Wang, to configure Reitan's method as claimed by presenting feedback indicating a spatial relationship between a position of the second annotation within the three- dimensional environment and the viewpoint of the user. The motivation is to enhance the apparent realistic of objects on the augmented reality space.
Claim 13 adds into claim 1 "while the computer system is participating in the communication session that includes the one or more participants, displaying, via the display, a representation of a first participant of the one or more participants of the communication session at a second position within the three-dimensional environment" (Reitan, [1099] - FIG. 13A shows an example interactive device 1310. Viewers 1312, 1314 and 1315 are able to watch content on the display 1311 of interactive device 1310. In various embodiments, content includes still video, still images, and/or audio. The content in FIG. 13A shows an office where one person is sitting at a desk and another person is sitting in a chair; [0974] - In an embodiment, an avatar 1101 is chosen based at least in part upon which user 1112, 1113, and 1114 is using the interactive device 1110. For example, a microphone may determine that a child is using the interactive device 1110 by the voice of the child and cause an avatar 1101 to appear wherein the avatar is a cartoon character. In an embodiment a microphone (e.g., by the number of voices) or a camera (e.g., by the number of bodies) may determine that a plurality of users 1112, 1113 and 1114 are using the interactive device 1110 and play content 1105 or choose an avatar 1101 in response to the particular users 1112, 1113, and 1114 that are present. In one embodiment, a plurality of avatars 1101 is shown concurrently).
Claim 14 adds into claim 1 "receiving, via the one or more input devices, second information, different from the first information, from a respective computer system, other than the computer system, that is communicating information to the computer system that is used to display the virtual scene at the computer system; and in response to receiving the second information, displaying, via the display, a second annotation at a respective position in the three-dimensional environment specified by the respective computer system" (Reitan, [1120] - In one embodiment, I/O device 1316 may receive dialogue from a plurality of users 1312, 1314, and 1315. Dialogue may comprise any speech, for example a discussion about a piece of clothing a woman is wearing. In an embodiment, when a discussion about an object 1301 is received from viewers 1312, 1314 and 1315 a processor 1317 may perform an operation (e.g., provide viewers 1312, 1314, and 1315 with additional content 1305). In an embodiment, a processor 1317 performs an operation based at least in part on the dialogue. For example, a processor 1317 may be operable to distinguish between different viewers 1312, 1314, and 1315; [0962] - Users 1112, 1113 and 1114 are shown watching the display 1111 of interactive device 1110. The display 1111 shows an example presentation layer (e.g., a layer displaying content 1105, interactive elements 1106, scroll bar 1107, and avatar 1101) Interactive device may include, but is not limited to: computers, televisions, radios, interactive televisions, video game consoles, mobile devices, smart phones, smart televisions, automobile consoles, windshields, laptops, personal digital assistants, tablet computers, etc) (Noted: each of the users 1112, 1113, and 1114 can access to the display 1111 through the interactive 1106 through its own computer for, e.g., placing objects on selected locations).
Claim 15 adds into claim 1 "wherein the correspondence between the first annotation and the respective position within the three-dimensional environment is based on a spatial relationship between one or more portions of the user's body relative to the three-dimensional environment" (Escudero, page 8, Spatial Awareness (see the video clip cited as https://learn.microsoft.com/shows/Docs-Miked-Reality/Microsofts- Designing-Holograms-Spatiel-Awareness-Chepter/player) - at 01:36-06:25, the placed location is defined from the user's body relative to the three-dimensional environment). Thus, it would have been obvious, in view of Escudero and Wang, to configure Reitan's method as claimed by presenting feedback indicating a spatial relationship between one or more portions of the user's body relative to the three-dimensional environment. The motivation is to enhance the apparent realistic of objects on the augmented reality space.
Claim 16 adds into claim 1 "wherein the context of the user's interaction with the three-dimensional environment is associated with one or more words spoken by the user of the computer system included in the first information" (Reitan, [0628] - In a second use case scenario, a smart T.V. with the system 600 coupled therewith enables voice interactivity via the T.V. user interface between one or more viewers of the T.V. and characters within the program being viewed on the T.V. A viewer of the T.V. program may speak with a character(s) within the T.V. program, while the context and meaning of the viewer's words and actions to the character(s) are interpreted via system 600).
Claim 17 adds into claim 1 "wherein the first position is a predetermined distance from the viewpoint of the user" (Escudero, page 8, Spatial Awareness (see the video clip cited as https://learn.microsoft.com/shows/Docs-Miked-Reality/Microsdfts- Designing-Holograms-Spatial-Awareness-Chapter/pleyer) - at 01:36-06:25, the placed location can be pre-defined by the user). Thus, it would have been obvious, in view of Escudero and Wang, to configure Reitan's method as claimed by pre-define the location having a specific distance from the user. The motivation is to enhance the apparent realistic of objects on the augmented reality space.
Claim 18 adds into claim 1 "while displaying the virtual scene from the viewpoint of the user, obtaining second information including a request to capture a two- dimensional image of the virtual scene, wherein a portion of the three-dimensional environment is visible via a viewport of the computer system when the second information is obtained" (Reitan, [1102] - computers and virtual reality allows advertisers to place objects 1301 into content (e.g., movies, slide shows, television programs, and video games) after the content is created with a high degree of realism. This is also known as retro-active product placement. For example, a system can place objects 1301 into a scene after it has been filmed; [1104] - While a television is shown as an example, in various embodiments interactive device 1310 may include, but is not limited to: a mobile device with a display 1311, a smart phone, a tablet computer, a laptop, a personal digital assistant, a smart television, a radio, a computer, a server, etc); and "in response to obtaining the second information, displaying, via the display, a virtual object including the two-dimensional image of the three-dimensional environment including the portion of the three-dimensional environment" (Reitan's placing of objects into a scene (e.g., [1102], [1103]) into a computer (e.g., mobile phone, tablet computer, ..) allow the system to capture and/or render a 2D image representing the 3D scene).
Claims 19 and 20 claim an electronic device and a non-transitory computer readable storage medium storing one or more programs based on the method of claims 1-18; therefore, they are rejected under a similar rationale.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHU K NGUYEN whose telephone number is (571)272-7645. The examiner can normally be reached M-F 8-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F. Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHU K NGUYEN/ Primary Examiner, Art Unit 2616
