DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 2-10, and 12-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 2-10, and 12-14 would be allowable over the prior art of record since the cited references taken individually or in combination fails to particularly disclose or suggest the limitations of each respective dependent claim, as presented in the environment of the remaining limitations of each respective dependent claim. It is noted that the closest prior art, Arora, shows the limitations of the parent claims from which each respective dependent claim depends. However, the Alora fails to disclose or suggest the specific limitations of each respective dependent claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 11, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Arora et al. (US Pub. 2021/0043005), hereinafter Arora, in view of Bar-Zeev et al. (US Pub. 2012/0127284), hereinafter Bar-Zeev.
Regarding claim 1, Arora discloses an information processing apparatus worn by a user, the information processing apparatus comprising: a processor controlling playback of a stereoscopic video virtual object (Fig. 6; Paragraph [0058]: GPU 420 can include a left channel output coupled to the left source of imagewise modulated light 424 and a right channel output coupled to the right source of imagewise modulated light 426. GPU 420 can output stereoscopic image data to the sources of imagewise modulated light 424, 426, for example as described above with respect to FIGS. 2A-2D); a positioning sensor detecting a position of the user (Fig. 5; Paragraph [0064]: FIG. 5 depicts a flow chart of an example usage of a SIG system. At step 502, a SIG recording session is initialized (e.g., a user can interact with a user interface to begin recording or set parameters for a recording session). At step 504, one or more SIG events are recorded. A SIG recording can incorporate data from one or more sensors on a wearable head device, which can include, e.g., a user's hand and head motions, audio signals, eye movement, location information, vital signs, or a spatial location); a gaze sensor detecting a direction of a line of sight of the user (Paragraph [0045]: and left and right eye cameras oriented toward the user (e.g., for detecting the user's eye movements; Paragraph [0064]: FIG. 5 depicts a flow chart of an example usage of a SIG system. At step 502, a SIG recording session is initialized (e.g., a user can interact with a user interface to begin recording or set parameters for a recording session). At step 504, one or more SIG events are recorded. A SIG recording can incorporate data from one or more sensors on a wearable head device, which can include, e.g., a user's hand and head motions, audio signals, eye movement); and a memory, wherein the processor records, in the memory, user information including at least one of a disposition position where the played video virtual object is disposed, the position of the user detected by the positioning sensor, and the direction of the line of sight of the user detected by the gaze sensor (Fig. 4; Paragraph [0059]: such as shown in FIG. 4, one or more of processor 416, GPU 420, DSP audio spatializer 422, HRTF memory 425, and audio/visual content memory 418 may be included in an auxiliary unit 400C (which may correspond to auxiliary unit 320 described above). The auxiliary unit 400C may include a battery 427 to power its components and/or to supply power to the wearable head device 400A or handheld controller 400B. Including such components in an auxiliary unit, which can be mounted to a user's waist, can limit the size and weight of the wearable head device 400A, which can in turn reduce fatigue of a user's head and neck; Paragraph [0096]: spatial content can be stored on a mixed reality system. In some embodiments, a mixed reality system can communicate with a server and transfer the SIG recording to a server. In some embodiments, a SIG recording can be stored on a mobile device as an intermediary between a mixed reality system and a server (e.g., a mixed reality system transfers a SIG recording to a mobile device, which then transfers a SIG recording to a server). In some embodiments, a mixed reality system is in communication with a mobile device during the SIG recording session such that the SIG recording is first stored on a mobile device. In some embodiments, a mixed reality system is in communication with a server during the SIG recording session such that the SIG recording is first stored on a server; Paragraph [0104]: the SIG recording's associated location can be the estimated initial location determined at step 1602. At step 1606, a mobile device can estimate its own location (e.g., during a playback session) using any suitable methods, including but not limited to: triangulation using wireless access points, GPS signals, cellular base stations, 5G base stations and/or recognized objects. Similarly, mixed reality system 603 can also be used to play back a SIG recording, and mixed reality system 603 can estimate its location using any suitable means. A new location estimate and a new accuracy estimate associated with the new location estimate can be stored on a mobile device, a mixed reality system, and/or on a server communicatively coupled to a mobile device or a mixed reality system), performs control to stop the playback of the video virtual object when detecting that the user has moved out of a range that is a predetermined distance away from the disposition position using the positioning sensor (Paragraph [0088]: a mixed reality system can display a virtual representation moving around a physical location 1310 (which can correspond to physical location 1204). In some embodiments, a virtual representation can mimic movements of a user who recorded a SIG event. For example, a mixed reality headset can track the position and orientation of a recording user's head, hands, and/or other body parts. As a recording user moves around a physical location and looks around, a virtual representation can be created that moves and looks around in an approximately similar manner as the recording user. In some embodiments, a mixed reality system can display a virtual representation 1304 creating virtual markings 1302 in an approximately similar manner as a recording user created virtual markings during a recording session. In some embodiments, a mixed reality system can display a virtual trail 1308. In some embodiments, a virtual trail indicates a previous location of a virtual representation. In some embodiments, a virtual trail can facilitate following a virtual representation around a physical location during a SIG playback session. In some embodiments, a SIG playback session can pause if a distance between a user and virtual content is greater than a customizable threshold. The threshold can be either static (e.g., a set distance in feet) or dynamic (e.g., varying with the size of a virtual object).
Alora does not explicitly disclose performs control to resume the playback of the video virtual object on the basis of the user information recorded in the memory and the position detected by the positioning sensor or the direction of the line of sight detected by the gaze sensor when the user enters the range after the playback of the video virtual object is stopped.
However, Bar-Zeev teaches augmented reality video in a head-mounted device (Abstract), further comprising performs control to resume the playback of the video virtual object on the basis of the user information recorded in the memory and the position detected by the positioning sensor or the direction of the line of sight detected by the gaze sensor when the user enters the range after the playback of the video virtual object is stopped (Paragraph [0164]: display of an augmented reality emitter can be triggered in response to determining that the user is looking at the video display screen, based on knowledge of an orientation and/or location of the user's head as obtained from one or more sensors of the HMD device. A determination that the user is looking at the video display screen can be made, e.g., by determining that the user is looking forward as opposed to looking up or down, and/or determining that the video display screen is within the user's field of view, or within a central, subset portion of the user's field of view. In this implementation, an augmented reality emitter is controlled to display augmented reality images which appear to the user to be in a static location in a substantially vertical plane, when the user looks at a video display screen, and the static location in the substantially vertical plane is a specified location relative to the video display screen, e.g., one foot to the right of the rightmost edge of the video display screen. When the user looks away from the video display screen, the display of the augmented reality image 1500 in a vertical plane at the static location with respect to the video display screen can be terminated. At this time, the augmented reality image 1500 could alternatively be displayed at another virtual location such as in a horizontal plane, as discussed below in connection with FIG. 15B, or not displayed at all. Subsequently, when the user again looks toward the video display screen, the display of the augmented reality image 1500 in the vertical plane at the static location with respect to the video display screen can be resumed; Paragraph [0171]: Step 1602 includes processing the image to identify outlines of objects in the scene (see also FIG. 11B). Optionally, if the camera has a depth sensing capability, a depth of the objects from the camera can be determined. Or, a depth camera of the hub can detect the distance of the user from the hub and communicate this distance to the HMD device as an indication of the distance from the HMD device to the video display screen, assuming the hub and the video display screen are approximately co-located. Step 1604 includes identifying one or more edges of the video display screen. Step 1606 includes determining an orientation and/or location of the user's head. For example, this can be done by determining an orientation and/or location of the HMD device based on one or more sensors carried by the HMD device, and/or by using information from a motion tracking depth camera at the hub). Bar-Zeev teaches that this will allow for rendering to a user in a realistic way for said user’s perspective (Paragraph [0149]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Alora with the features of above as taught by Bar-Zeev so as to allow for rendering to a user in a realistic way for said user’s perspective as presented by Bar-Zeev.
Regarding claim 11, Arora discloses an information processing apparatus that has a function of generating a stereoscopic video virtual object and displaying the stereoscopic video virtual object and is worn by a user (Fig. 6; Paragraph [0058]: GPU 420 can include a left channel output coupled to the left source of imagewise modulated light 424 and a right channel output coupled to the right source of imagewise modulated light 426. GPU 420 can output stereoscopic image data to the sources of imagewise modulated light 424, 426, for example as described above with respect to FIGS. 2A-2D), the information processing apparatus comprising: a processor (Paragraph [0058]: GPU 420 can include a left channel output coupled to the left source of imagewise modulated light 424 and a right channel output coupled to the right source of imagewise modulated light 426. GPU 420 can output stereoscopic image data to the sources of imagewise modulated light 424, 426, for example as described above with respect to FIGS. 2A-2D); and a storage (Fig. 4; Paragraph [0059]: such as shown in FIG. 4, one or more of processor 416, GPU 420, DSP audio spatializer 422, HRTF memory 425, and audio/visual content memory 418 may be included in an auxiliary unit 400C (which may correspond to auxiliary unit 320 described above). The auxiliary unit 400C may include a battery 427 to power its components and/or to supply power to the wearable head device 400A or handheld controller 400B. Including such components in an auxiliary unit, which can be mounted to a user's waist, can limit the size and weight of the wearable head device 400A, which can in turn reduce fatigue of a user's head and neck), wherein the processor records mutual disposition information, which is a mutual position and direction relationship between the video virtual object and a position and a line of sight of the user, in the storage (Fig. 4; Paragraph [0059]: such as shown in FIG. 4, one or more of processor 416, GPU 420, DSP audio spatializer 422, HRTF memory 425, and audio/visual content memory 418 may be included in an auxiliary unit 400C (which may correspond to auxiliary unit 320 described above). The auxiliary unit 400C may include a battery 427 to power its components and/or to supply power to the wearable head device 400A or handheld controller 400B. Including such components in an auxiliary unit, which can be mounted to a user's waist, can limit the size and weight of the wearable head device 400A, which can in turn reduce fatigue of a user's head and neck; Paragraph [0096]: spatial content can be stored on a mixed reality system. In some embodiments, a mixed reality system can communicate with a server and transfer the SIG recording to a server. In some embodiments, a SIG recording can be stored on a mobile device as an intermediary between a mixed reality system and a server (e.g., a mixed reality system transfers a SIG recording to a mobile device, which then transfers a SIG recording to a server). In some embodiments, a mixed reality system is in communication with a mobile device during the SIG recording session such that the SIG recording is first stored on a mobile device. In some embodiments, a mixed reality system is in communication with a server during the SIG recording session such that the SIG recording is first stored on a server; Paragraph [0104]: the SIG recording's associated location can be the estimated initial location determined at step 1602. At step 1606, a mobile device can estimate its own location (e.g., during a playback session) using any suitable methods, including but not limited to: triangulation using wireless access points, GPS signals, cellular base stations, 5G base stations and/or recognized objects. Similarly, mixed reality system 603 can also be used to play back a SIG recording, and mixed reality system 603 can estimate its location using any suitable means. A new location estimate and a new accuracy estimate associated with the new location estimate can be stored on a mobile device, a mixed reality system, and/or on a server communicatively coupled to a mobile device or a mixed reality system), stops playback of the video virtual object when the user moves away from the video virtual object and a distance between the video virtual object and the user is equal to or greater than a predetermined distance (Paragraph [0088]: a mixed reality system can display a virtual representation moving around a physical location 1310 (which can correspond to physical location 1204). In some embodiments, a virtual representation can mimic movements of a user who recorded a SIG event. For example, a mixed reality headset can track the position and orientation of a recording user's head, hands, and/or other body parts. As a recording user moves around a physical location and looks around, a virtual representation can be created that moves and looks around in an approximately similar manner as the recording user. In some embodiments, a mixed reality system can display a virtual representation 1304 creating virtual markings 1302 in an approximately similar manner as a recording user created virtual markings during a recording session. In some embodiments, a mixed reality system can display a virtual trail 1308. In some embodiments, a virtual trail indicates a previous location of a virtual representation. In some embodiments, a virtual trail can facilitate following a virtual representation around a physical location during a SIG playback session. In some embodiments, a SIG playback session can pause if a distance between a user and virtual content is greater than a customizable threshold. The threshold can be either static (e.g., a set distance in feet) or dynamic (e.g., varying with the size of a virtual object).
Alora does not explicitly disclose disposes the video virtual object so as to follow a movement of the information processing apparatus in a state based on the mutual disposition information recorded in the storage after a predetermined time has elapsed since the stop; and controls a playback operation of the disposed video virtual object.
However, Bar-Zeev teaches augmented reality video in a head-mounted device (Abstract), further comprising disposes the video virtual object so as to follow a movement of the information processing apparatus in a state based on the mutual disposition information recorded in the storage after a predetermined time has elapsed since the stop (Paragraph [0164]: display of an augmented reality emitter can be triggered in response to determining that the user is looking at the video display screen, based on knowledge of an orientation and/or location of the user's head as obtained from one or more sensors of the HMD device. A determination that the user is looking at the video display screen can be made, e.g., by determining that the user is looking forward as opposed to looking up or down, and/or determining that the video display screen is within the user's field of view, or within a central, subset portion of the user's field of view. In this implementation, an augmented reality emitter is controlled to display augmented reality images which appear to the user to be in a static location in a substantially vertical plane, when the user looks at a video display screen, and the static location in the substantially vertical plane is a specified location relative to the video display screen, e.g., one foot to the right of the rightmost edge of the video display screen. When the user looks away from the video display screen, the display of the augmented reality image 1500 in a vertical plane at the static location with respect to the video display screen can be terminated. At this time, the augmented reality image 1500 could alternatively be displayed at another virtual location such as in a horizontal plane, as discussed below in connection with FIG. 15B, or not displayed at all. Subsequently, when the user again looks toward the video display screen, the display of the augmented reality image 1500 in the vertical plane at the static location with respect to the video display screen can be resumed; Paragraph [0171]: Step 1602 includes processing the image to identify outlines of objects in the scene (see also FIG. 11B). Optionally, if the camera has a depth sensing capability, a depth of the objects from the camera can be determined. Or, a depth camera of the hub can detect the distance of the user from the hub and communicate this distance to the HMD device as an indication of the distance from the HMD device to the video display screen, assuming the hub and the video display screen are approximately co-located. Step 1604 includes identifying one or more edges of the video display screen. Step 1606 includes determining an orientation and/or location of the user's head. For example, this can be done by determining an orientation and/or location of the HMD device based on one or more sensors carried by the HMD device, and/or by using information from a motion tracking depth camera at the hub; Paragraph [0172]: Step 1608 includes storing data representing the edges of the video display screen. For example, each edge can be represented by a vector having defined endpoints. It may be sufficient to determine the edges of the video display screen one time and to use that information in subsequent viewing by the user. Or, the edges of the video display screen can be re-determined at the start of a viewing session by the user, or when the HMD device is powered on, or once a day, or at other specified times, or when a specified movement or change in location of the user is detected, or based on other triggering criteria); and controls a playback operation of the disposed video virtual object (Fig. 15B; Paragraphs [0164]-[0165]: determination that the user is looking at the video display screen can be made, e.g., by determining that the user is looking forward as opposed to looking up or down, and/or determining that the video display screen is within the user's field of view, or within a central, subset portion of the user's field of view. In this implementation, an augmented reality emitter is controlled to display augmented reality images which appear to the user to be in a static location in a substantially vertical plane, when the user looks at a video display screen, and the static location in the substantially vertical plane is a specified location relative to the video display screen, e.g., one foot to the right of the rightmost edge of the video display screen. When the user looks away from the video display screen, the display of the augmented reality image 1500 in a vertical plane at the static location with respect to the video display screen can be terminated. At this time, the augmented reality image 1500 could alternatively be displayed at another virtual location such as in a horizontal plane, as discussed below in connection with FIG. 15B, or not displayed at all. Subsequently, when the user again looks toward the video display screen, the display of the augmented reality image 1500 in the vertical plane at the static location with respect to the video display screen can be resumed….depicts an augmented reality image which is in a static location on a substantially horizontal plane when a user looks down. Here, assume the user 1510 is in a position in which he can look forward to see the cooking show on the video display screen 1110 of FIG. 15A, and also look down at a table 1514 with a generally horizontal surface, e.g., where the user works to follow the recipe of the cooking show. The user periodically looks down, as indicated by the downward angle .alpha. of the HMD device 1512 relative to the horizontal. In response to detecting that the user is looking down, the augmented reality image can be made to appear to the user to be in a static location in a substantially horizontal plane, such as depicted by the augmented reality image 1516 on the table 1514. In this way, the user can easy follow the text or other instructions while transitioning between looking forward at the video display screen and looking down at the work surface. In addition to a cooking show, the augmented reality image can provide instructions or other information which supplements, and is related to, the content on the video display screen. Related content could also include a director's commentary or an actor's interview relating to a movie which is provided on the video display screen). Bar-Zeev teaches that this will allow for rendering to a user in a realistic way for said user’s perspective (Paragraph [0149]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Alora with the features of above as taught by Bar-Zeev so as to allow for rendering to a user in a realistic way for said user’s perspective as presented by Bar-Zeev.
Regarding claim 15, Alora discloses an information processing method in an apparatus that is worn by a user and is used, the information processing method comprising: playing back a stereoscopic video virtual object (Fig. 6; Paragraph [0058]: GPU 420 can include a left channel output coupled to the left source of imagewise modulated light 424 and a right channel output coupled to the right source of imagewise modulated light 426. GPU 420 can output stereoscopic image data to the sources of imagewise modulated light 424, 426, for example as described above with respect to FIGS. 2A-2D); disposing and displaying the played video virtual object to be superimposed on a real object (Paragraph [0006]: AR systems present a virtual environment that overlaps or overlays the real environment in at least one aspect. For example, an AR system could present the user with a view of a virtual environment overlaid on the user's view of the real environment, such as with a transmissive head-mounted display that presents a displayed image while allowing light to pass through the display into the user's eye. Similarly, an AR system could present the user with audio corresponding to the virtual environment, while simultaneously mixing in audio from the real environment. Similarly, as used herein, MR systems present a virtual environment that overlaps or overlays the real environment in at least one aspect, as do AR systems, and may additionally allow that a virtual environment in an MR system may interact with the real environment in at least one aspect; Paragraph [0075]: a user can depress and hold a button on a handheld controller 904 to “write” with a virtual marker 906. In some embodiments, a virtual marker 906 can be displayed during a recording session and overlaid on a handheld controller 904 when the handheld controller comes into a user's field of view. In some embodiments, a virtual controller and/or hand can be displayed (e.g., when a virtual marker is not engaged) during a recording session and overlaid on a handheld controller when the handheld controller comes into a user's field of view. In some embodiments, when a marker function is active, a user can use their hands to create virtual markings 902. In some embodiments, a mixed reality system can track a user's hand movement through various sensors (e.g., depth sensors, RGB cameras, other optical sensors, or motion sensors). In some embodiments, a user can activate a marker function by pointing with the user's hand. A virtual marker can be displayed and optionally overlaid on a user's hand when the hand is in a user's field of view to indicate a marker function is active); recording user information including at least one of a disposition position where the played video virtual object is disposed, a position of the user (Fig. 4; Paragraph [0059]: such as shown in FIG. 4, one or more of processor 416, GPU 420, DSP audio spatializer 422, HRTF memory 425, and audio/visual content memory 418 may be included in an auxiliary unit 400C (which may correspond to auxiliary unit 320 described above). The auxiliary unit 400C may include a battery 427 to power its components and/or to supply power to the wearable head device 400A or handheld controller 400B. Including such components in an auxiliary unit, which can be mounted to a user's waist, can limit the size and weight of the wearable head device 400A, which can in turn reduce fatigue of a user's head and neck; Paragraph [0096]: spatial content can be stored on a mixed reality system. In some embodiments, a mixed reality system can communicate with a server and transfer the SIG recording to a server. In some embodiments, a SIG recording can be stored on a mobile device as an intermediary between a mixed reality system and a server (e.g., a mixed reality system transfers a SIG recording to a mobile device, which then transfers a SIG recording to a server). In some embodiments, a mixed reality system is in communication with a mobile device during the SIG recording session such that the SIG recording is first stored on a mobile device. In some embodiments, a mixed reality system is in communication with a server during the SIG recording session such that the SIG recording is first stored on a server; Paragraph [0104]: the SIG recording's associated location can be the estimated initial location determined at step 1602. At step 1606, a mobile device can estimate its own location (e.g., during a playback session) using any suitable methods, including but not limited to: triangulation using wireless access points, GPS signals, cellular base stations, 5G base stations and/or recognized objects. Similarly, mixed reality system 603 can also be used to play back a SIG recording, and mixed reality system 603 can estimate its location using any suitable means. A new location estimate and a new accuracy estimate associated with the new location estimate can be stored on a mobile device, a mixed reality system, and/or on a server communicatively coupled to a mobile device or a mixed reality system), and a direction of a line of sight of the user (Paragraph [0045]: and left and right eye cameras oriented toward the user (e.g., for detecting the user's eye movements; Paragraph [0064]: FIG. 5 depicts a flow chart of an example usage of a SIG system. At step 502, a SIG recording session is initialized (e.g., a user can interact with a user interface to begin recording or set parameters for a recording session). At step 504, one or more SIG events are recorded. A SIG recording can incorporate data from one or more sensors on a wearable head device, which can include, e.g., a user's hand and head motions, audio signals, eye movement); stopping the playback of the video virtual object when it is detected that the user has moved out of a range that is a predetermined distance away from the disposition position (Paragraph [0088]: a mixed reality system can display a virtual representation moving around a physical location 1310 (which can correspond to physical location 1204). In some embodiments, a virtual representation can mimic movements of a user who recorded a SIG event. For example, a mixed reality headset can track the position and orientation of a recording user's head, hands, and/or other body parts. As a recording user moves around a physical location and looks around, a virtual representation can be created that moves and looks around in an approximately similar manner as the recording user. In some embodiments, a mixed reality system can display a virtual representation 1304 creating virtual markings 1302 in an approximately similar manner as a recording user created virtual markings during a recording session. In some embodiments, a mixed reality system can display a virtual trail 1308. In some embodiments, a virtual trail indicates a previous location of a virtual representation. In some embodiments, a virtual trail can facilitate following a virtual representation around a physical location during a SIG playback session. In some embodiments, a SIG playback session can pause if a distance between a user and virtual content is greater than a customizable threshold. The threshold can be either static (e.g., a set distance in feet) or dynamic (e.g., varying with the size of a virtual object).
Alora does not explicitly disclose resuming the playback of the video virtual object on the basis of the recorded user information when the user enters the range after the playback of the video virtual object is stopped.
However, Bar-Zeev teaches augmented reality video in a head-mounted device (Abstract), further comprising resuming the playback of the video virtual object on the basis of the recorded user information when the user enters the range after the playback of the video virtual object is stopped (Paragraph [0164]: display of an augmented reality emitter can be triggered in response to determining that the user is looking at the video display screen, based on knowledge of an orientation and/or location of the user's head as obtained from one or more sensors of the HMD device. A determination that the user is looking at the video display screen can be made, e.g., by determining that the user is looking forward as opposed to looking up or down, and/or determining that the video display screen is within the user's field of view, or within a central, subset portion of the user's field of view. In this implementation, an augmented reality emitter is controlled to display augmented reality images which appear to the user to be in a static location in a substantially vertical plane, when the user looks at a video display screen, and the static location in the substantially vertical plane is a specified location relative to the video display screen, e.g., one foot to the right of the rightmost edge of the video display screen. When the user looks away from the video display screen, the display of the augmented reality image 1500 in a vertical plane at the static location with respect to the video display screen can be terminated. At this time, the augmented reality image 1500 could alternatively be displayed at another virtual location such as in a horizontal plane, as discussed below in connection with FIG. 15B, or not displayed at all. Subsequently, when the user again looks toward the video display screen, the display of the augmented reality image 1500 in the vertical plane at the static location with respect to the video display screen can be resumed; Paragraph [0171]: Step 1602 includes processing the image to identify outlines of objects in the scene (see also FIG. 11B). Optionally, if the camera has a depth sensing capability, a depth of the objects from the camera can be determined. Or, a depth camera of the hub can detect the distance of the user from the hub and communicate this distance to the HMD device as an indication of the distance from the HMD device to the video display screen, assuming the hub and the video display screen are approximately co-located. Step 1604 includes identifying one or more edges of the video display screen. Step 1606 includes determining an orientation and/or location of the user's head. For example, this can be done by determining an orientation and/or location of the HMD device based on one or more sensors carried by the HMD device, and/or by using information from a motion tracking depth camera at the hub). Bar-Zeev teaches that this will allow for rendering to a user in a realistic way for said user’s perspective (Paragraph [0149]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Alora with the features of above as taught by Bar-Zeev so as to allow for rendering to a user in a realistic way for said user’s perspective as presented by Bar-Zeev.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Benedetto et al. (US Pub. 2017/0354883) teaches HMD with active zone control according to user gaze.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW D SALVUCCI whose telephone number is (571)270-5748. The examiner can normally be reached M-F: 7:30-4:00PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW SALVUCCI/Primary Examiner, Art Unit 2613
Prosecution Insights