DETAILED ACTION
This action is in response to the application filed 05/07/2024. Claims 1 – 20 are pending and have
been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 7, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Carteri et al. (U.S. Pub. No. 2021/0271530, hereinafter “Carteri”) in view of Baron, JR. et al. (U.S. Pub. No. 2012/0050323, hereinafter “Baron”).
Regarding Claim 1, Carteri teaches
A system (see Carteri Abstract, system) comprising:
control circuitry (see Carteri Paragraph [0043], For example, in some aspects, a statistics (stats) engine 252 of system architecture 200 may be performed by browser 202. In some aspects, a user interface (UI) optimizer 260 of system architecture 200 may be performed by cloud/enterprise 204, Paragraph [0069], In some aspects, performance monitor 234 may collect or receive real time performance data and statistics information about the widgets (e.g., visual elements) of the UI. Performance monitor 234 may provide or transmit 235 the alerts, kpi, performance statistics, UI activities, or other similar data about the usage and performance of browser 202 and cloud/enterprise 204 to a stats engine 252 for further processing. Stats engine 252 may, for example, be implemented by cloud/enterprise 204 and may be configured to transmit 239 the usage and performance data to database 240 for storage and later use. Stats engine 252 may also be configured to receive any of the above mentioned data stored in database 240 for further analysis, Paragraph [0070], In some aspects, for example, stats engine 252 may be configured to analyze the alerts, kpi, performance status, UI activities, or other data received from performance monitor 234, databases 240, or from other sources, to determine performance weights of tasks 216 and widgets 212 or perform other performance related analysis of the received data. For example, stats engine 252 may collect metrics for each scenario step, e.g., response time, CPU usage, network usage, etc., from performance monitor 234 and database 240 and may aggregate the metrics along time, and Paragraph [0071], In some aspects, the performance data of the widgets 212 may also be analyzed by the stats engine 252 to assign a relative performance weight to each widget) configured to:
determine a capability of a device (see Carteri Paragraph [0043], For example, in some aspects, a statistics (stats) engine 252 of system architecture 200 may be performed by browser 202. In some aspects, a user interface (UI) optimizer 260 of system architecture 200 may be performed by cloud/enterprise 204, Paragraph [0069], In some aspects, performance monitor 234 may collect or receive real time performance data and statistics information about the widgets (e.g., visual elements) of the UI. Performance monitor 234 may provide or transmit 235 the alerts, kpi, performance statistics, UI activities, or other similar data about the usage and performance of browser 202 and cloud/enterprise 204 to a stats engine 252 for further processing. Stats engine 252 may, for example, be implemented by cloud/enterprise 204 and may be configured to transmit 239 the usage and performance data to database 240 for storage and later use. Stats engine 252 may also be configured to receive any of the above mentioned data stored in database 240 for further analysis, Paragraph [0070], In some aspects, for example, stats engine 252 may be configured to analyze the alerts, kpi, performance status, UI activities, or other data received from performance monitor 234, databases 240, or from other sources, to determine performance weights of tasks 216 and widgets 212 or perform other performance related analysis of the received data. For example, stats engine 252 may collect metrics for each scenario step, e.g., response time, CPU usage, network usage, etc., from performance monitor 234 and database 240 and may aggregate the metrics along time, and Paragraph [0071], In some aspects, the performance data of the widgets 212 may also be analyzed by the stats engine 252 to assign a relative performance weight to each widget, in which the device capability is performance capability); and
input/output circuitry (see Carteri Paragraph [0091], In some aspects, stats engine 252 may transmit stats 253 and alerts to a UI optimizer 260 to trigger 262 a modification of the UI when a current or future degradation in performance of the UI is detected, for example, performance of the UI has degraded or fallen below a threshold amount, performance of the UI is degrading or decreasing faster than a threshold rate, etc. and Paragraph [0092], In response to receiving performance alerts from stats engine 252, UI optimizer 260 may modify 262 the UI to reduce the required resources in an attempt to improve or maintain performance of the UI. The goal of the modification may be to maintain a sufficient amount of capability in the UI so that the user may still be able to accomplish the task while minimizing the impact of the modification on the usability of the UI, for example, by using profiling and preferences data for the user) configured to:
based at least in part on the determined capability of the device, generate for display a user interface preventing selection of an option (see Carteri Paragraph [0091], In some aspects, stats engine 252 may transmit stats 253 and alerts to a UI optimizer 260 to trigger 262 a modification of the UI when a current or future degradation in performance of the UI is detected, for example, performance of the UI has degraded or fallen below a threshold amount, performance of the UI is degrading or decreasing faster than a threshold rate, etc. Paragraph [0092], In response to receiving performance alerts from stats engine 252, UI optimizer 260 may modify 262 the UI to reduce the required resources in an attempt to improve or maintain performance of the UI. The goal of the modification may be to maintain a sufficient amount of capability in the UI so that the user may still be able to accomplish the task while minimizing the impact of the modification on the usability of the UI, for example, by using profiling and preferences data for the user, and Paragraph [0093], In some aspects, the modification 262 may include disabling a widget in the UI. In some aspects, a disabled widget may be removed from display in the UI. In some aspects, a disabled widget may still be illustrated in the UI but its underlying features, service calls, and corresponding API's may not be executed or activated. In some aspects, only a portion of the widget's functionality may be disabled, e.g., some but not all of the services may be disabled. In this manner, resources associated with the disabled widget or widget function are not allocated and the overall resource usage of the UI may be reduced)
Carteri does not expressively teach
to indicate that one or more entities should not be occluded by an XR portion configured to be inserted in at least one portion of one or more frames corresponding to a physical background of an environment of a user associated with the device.
However, Baron teaches
to indicate that one or more entities should not be occluded by an XR portion configured to be inserted in at least one portion of one or more frames corresponding to a physical background of an environment of a user associated with the device (see Baron Paragraph [0006], configuring display of a first videoconference participant to a second videoconference participant of the videoconference using a computer system, the configuring including: selecting a first virtual background for the first videoconference participant from a plurality of virtual backgrounds based on at least one of: a first videoconference participant identification or a second videoconference participant identification by applying a set of virtual background selection rules to the at least one of: the first videoconference participant identification or the second videoconference participant identification using the computer system; and providing the first virtual background from the computer system for display to the second videoconference participant in conjunction with isolated video data corresponding to the first videoconference participant isolated from an actual background of the first videoconference participant during the videoconference, and Paragraph [0033], In any event, the image data 44 can be processed to isolate the participant image data 46 from the background image data 48 using any solution. In an embodiment, the user system 12 includes isolation software, which is configured to identify and isolate the participant image data 46 in a video stream from the background image data 48. In an alternative embodiment, the user system 12 can provide video data including all of the image data 44 for processing by the computer system 20, which in turn isolates the participant image data 46 from the background image data 48 using any solution, in which the isolating of the participant indicates to not occlude a portion of the video frame).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system in which, based on a capability of a device, generates for display a user interface preventing selection of an option (as taught in Carteri), with indicating one or more entities should not be occluded by an XR portion configured to be inserted in a portion of one or more frames corresponding to a physical background of an environment of a user associated with a device (as taught in Baron), the motivation being to create an adaptive UI rendering that reduces likelihood of distractions (see Baron Paragraph [0032]).
Regarding Claim 2, Carteri in view of Baron teaches
The system of claim 1, wherein the input/output circuitry is configured to generate for display the user interface preventing selection of the option by graying out the option (see Carteri Paragraph [0087], During a performance degradation, the wizard may be reduced to only presenting the open ticket panel and may not present the search panel to the user. In some aspects, the search panel may alternatively be presented but temporarily disabled, e.g., greyed our or in some other form of disablement).
Regarding Claim 7, Carteri in view of Baron teaches
The system of claim 1, wherein the determined capability is that the device can only perform a subset of options provided by the user interface (see Carteri Paragraph [0091], In some aspects, stats engine 252 may transmit stats 253 and alerts to a UI optimizer 260 to trigger 262 a modification of the UI when a current or future degradation in performance of the UI is detected, for example, performance of the UI has degraded or fallen below a threshold amount, performance of the UI is degrading or decreasing faster than a threshold rate, etc. Paragraph [0092], In response to receiving performance alerts from stats engine 252, UI optimizer 260 may modify 262 the UI to reduce the required resources in an attempt to improve or maintain performance of the UI. The goal of the modification may be to maintain a sufficient amount of capability in the UI so that the user may still be able to accomplish the task while minimizing the impact of the modification on the usability of the UI, for example, by using profiling and preferences data for the user, and Paragraph [0093], In some aspects, the modification 262 may include disabling a widget in the UI. In some aspects, a disabled widget may be removed from display in the UI. In some aspects, a disabled widget may still be illustrated in the UI but its underlying features, service calls, and corresponding API's may not be executed or activated. In some aspects, only a portion of the widget's functionality may be disabled, e.g., some but not all of the services may be disabled. In this manner, resources associated with the disabled widget or widget function are not allocated and the overall resource usage of the UI may be reduced).
Regarding Claims 17 - 18, they are rejected similarly as Claims 1 - 2, respectively. The method can be found in Carteri (Abstract, method).
Claims 3 – 5, 8 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Carteri et al. (U.S. Pub. No. 2021/0271530, hereinafter “Carteri”) in view of Baron, JR. et al. (U.S. Pub. No. 2012/0050323, hereinafter “Baron”) and Wang et al. (U.S. Pub. No. 2021/0026593, hereinafter “Wang”).
Regarding Claim 3, Carteri in view of Baron teaches all the limitations of claim 1, but does not expressively teach
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by causing the device to transmit an indication of a type of device.
However, Wang teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by causing the device to transmit an indication of a type of device (see Wang Paragraph [0041], Device model data can include, for example, an indicator (or indication) of the device model (e.g., a device model identifier or a device model name), indications of properties, characteristics, features or capabilities of devices associated with the device model, a name of the corresponding second-party device provider, a description of devices associated with the device model, and indication of a device type, or a combination thereof).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system in which, based on a capability of a device, generates for display a user interface preventing selection of an option indicating one or more entities should not be occluded by an XR portion configured to be inserted in a portion of one or more frames corresponding to a physical background of an environment of a user associated with a device (as taught in Carteri in view of Baron), with determining capability of a device by causing the device to transmit an indication of a type of device (as taught in Wang), the motivation being to reduce processing, power consumption, and bandwidth usage by recognizing a type of device which could provide pre-defined information (see Wang Paragraphs [0034] and [0041]).
Regarding Claim 4, Carteri in view of Baron and Wang teaches
The system of claim 3, wherein the indication of the type of device is a device identifier (see Wang Paragraph [0041], Device model data can include, for example, an indicator (or indication) of the device model (e.g., a device model identifier or a device model name), indications of properties, characteristics, features or capabilities of devices associated with the device model, a name of the corresponding second-party device provider, a description of devices associated with the device model, and indication of a device type, or a combination thereof).
Regarding Claim 5, Carteri in view of Baron and Wang teaches
The system of claim 4, wherein the device identifier comprises an attribute comprising of at least one of a device name, a device type, a model number, a serial number, a manufacturer name, or a battery life (see Wang Paragraph [0041], Device model data can include, for example, an indicator (or indication) of the device model (e.g., a device model identifier or a device model name), indications of properties, characteristics, features or capabilities of devices associated with the device model, a name of the corresponding second-party device provider, a description of devices associated with the device model, and indication of a device type, or a combination thereof).
Regarding Claim 8, Carteri in view of Baron and Wang teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining a type of device (see Wang Paragraph [0041], Device model data can include, for example, an indicator (or indication) of the device model (e.g., a device model identifier or a device model name), indications of properties, characteristics, features or capabilities of devices associated with the device model, a name of the corresponding second-party device provider, a description of devices associated with the device model, and indication of a device type, or a combination thereof).
Regarding Claim 19, it is rejected similarly as Claim 3. The method can be found in Carteri (Abstract, method).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Carteri et al. (U.S. Pub. No. 2021/0271530, hereinafter “Carteri”) in view of Baron, JR. et al. (U.S. Pub. No. 2012/0050323, hereinafter “Baron”) and Taylor et al. (EP Pub. No. 4472246, hereinafter “Taylor”).
Regarding Claim 6, Carteri in view of Baron teaches all the limitations of claim 1, but does not expressively teach
The system of claim 1, wherein the determined capability is that the device is not capable of performing processing to insert the XR portion.
However, Taylor teaches
The system of claim 1, wherein the determined capability is that the device is not capable of performing processing to insert the XR portion (see Taylor Paragraph [0030], The remote UI information that is provided from the electronic device 105 to electronic device 205 may include one or more images, one or more video streams, and/or one or more other assets associated with one or more elements of the user interface, one or more layer trees that describe the layout and/or appearance of the one or elements of the user interface, and metadata for the user interface. In one or more implementations, the metadata can include the one or more layer trees. For example, when the rendering engine 223 generates rendered display frames for display of the UI by display 225, the rendering engine 223 may also generate one or more display frames of portions of the UI (e.g., elements of the UI such as text fields, buttons, tools, windows, dynamic content, images, embedded videos, etc.), Paragraph [0034], In some use cases, the electronic device 105 may determine that the version of the UI that is transmitted for rendering at the electronic device 205 can no longer be transmitted as a combination asset(s), layer tree(s), and/or metadata. For example, the electronic device 105 may determine that the bandwidth and/or quality of the connection between the electronic device 105 and the electronic device 205 is insufficient, that computing resources (e.g., power, memory, and/or processing resources) of the receiving device are insufficient, and/or that compatibility issue between the operating systems of the electronic device 105 and the electronic device 205 may lead to and/or may have led to an invalid state of the UI, or a portion thereof, at the electronic device 205. In a use case in which the electronic device 105 determines that the version of the UI that is transmitted for rendering at the electronic device 205 can no longer be transmitted as a combination asset(s), layer tree(s), and/or metadata (e.g., due to a reduced capability of the other electronic device 105), the electronic device may switch to a fallback mode in which the UI (e.g., the entire UI) is encoded into a video stream and transmitted to the electronic device 205 (e.g., without metadata for reconstruction and/or rendering of the UI at the electronic device 205), and Paragraph [0041], OS service 260 may be a service that is provided by the operating system of the electronic device 205, and that performs operations for generating an XR environment. In one or more implementations, the anchoring information may be serialized, along with one or more assets (e.g., images, video streams, etc.), one or more layer trees, and/or other metadata (e.g., including timing information) for transmission from the electronic device 105 to the electronic device 205).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system in which, based on a capability of a device, generates for display a user interface preventing selection of an option indicating one or more entities should not be occluded by an XR portion configured to be inserted in a portion of one or more frames corresponding to a physical background of an environment of a user associated with a device (as taught in Carteri in view of Baron), with determining that a device is not capable of performing processing to insert a XR portion (as taught in Taylor), the motivation being to identify an issue (such as insufficient bandwidth or connection quality) and provide a solution to maintain a video conference stream of the user (see Taylor Paragraph [0034]).
Claims 9, 11, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Carteri et al. (U.S. Pub. No. 2021/0271530, hereinafter “Carteri”) in view of Baron, JR. et al. (U.S. Pub. No. 2012/0050323, hereinafter “Baron”) and Marshall et al. (U.S. Pub. No. 2022/0303185, hereinafter “Marshall”).
Regarding Claim 9, Carteri in view of Baron teaches all the limitations of claim 1, but does not expressively teach
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining whether or not the device can output audio and/or video.
However, Marshall teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining whether or not the device can output audio and/or video (see Marshall Paragraph [0093], At block 704, the first user device can derive a first set of device capabilities that correspond to the first user device. The first set of device capabilities can specify various capabilities of the first user device. For instance, deriving the first set of device capabilities can include identifying a series of features unique to the first user device, identifying types of data capable of being outputted by the first user device, Paragraph [0053], identifying capabilities specific to each device (e.g., whether a device can output audio content, whether a device can stream video content)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system in which, based on a capability of a device, generates for display a user interface preventing selection of an option indicating one or more entities should not be occluded by an XR portion configured to be inserted in a portion of one or more frames corresponding to a physical background of an environment of a user associated with a device (as taught in Carteri in view of Baron), with determining a device’s capability by determining whether or not the device can output audio and/or video (as taught in Marshall), the motivation being to enable features only when they are supported by a device, therefore preventing issues (see Marshall Paragraph [0017]).
Regarding Claim 11, Carteri in view of Baron and Marshall teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining a power connection of the device (see Marshall Paragraph [0035], The devices in the cluster can include a unique ID that specifies the device and characteristics of the device (e.g., a device type, device class, whether the device is wired or wireless, battery powered, etc., Paragraph [0018], The first user device can derive a set of device capabilities for the first user device. The device capabilities for the first user device can specify various capabilities (e.g., power capabilities, processing capabilities, network capabilities, features of the device, etc.) specific to the first user device, and Paragraph [0061], Device capability data can include data identifying capabilities of each device. For example, the device capability data can specify a processing capability, a power capability, a network capability, a device class, a software version of the device, whether the device is battery powered (e.g., a power configuration)).
Regarding Claim 13, Carteri in view of Baron and Marshall teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining whether the device is a mobile device (see Marshall Paragraph [0024], For example, the cluster capability information can specify types of devices in the first cluster (e.g., mobile phones, video-enabled devices, devices with audio capabilities)).
Regarding Claim 14, Carteri in view of Baron and Marshall teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining whether a resolution is supported by the device (see Marshall Paragraph [0067], The leader user device can process the request for a service user device from the service user device node and identify a series of parameters specific to the request. The parameters can specify a requested data type (e.g., video, audio), a requested processing power of devices in the network, a requested network throughput capability of devices in the network, etc. In the example of a request requesting a video stream from a device in a 4K (3840×2160 pixel), the leader device 1 404 can compare cluster capability information for each cluster to determine that the second cluster has a device capable of streaming a video in the 4K resolution. In response, the leader user device 404 can forward the second cluster capability information to the service user device as described in block 436 below).
Claims 10, 12, 15 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Carteri et al. (U.S. Pub. No. 2021/0271530, hereinafter “Carteri”) in view of Baron, JR. et al. (U.S. Pub. No. 2012/0050323, hereinafter “Baron”) and Zavesky et al. (U.S. Pub. No. 2021/0185222, hereinafter “Zavesky”).
Regarding Claim 10, Carteri in view of Baron teaches all the limitations of claim 1, but does not expressively teach
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining how quickly content is downloaded to the device.
However, Zavesky teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining how quickly content is downloaded to the device (see Zavesky Paragraph [0005], One or more device capabilities can be known or can be determined for the user device or other device(s) being used to capture one or more instances of media content for the project. Thus, for example, the device capabilities can define a maximum resolution for video that is captured by the user device, a data transfer rate attainable and/or sustainable for the user device, data processing and/or storage capabilities, combinations thereof, or the like, Paragraph [0039], The device capabilities 120 can define, for example, one or more communication capabilities of the user device 102 such as, for example, an average and/or peak data transmission rate, an average and/or peak data download rate, and Paragraph [0069], In various example embodiments, the device capabilities can represent battery life of the user device 102, memory capacity of the user device 102, processing speed and/or other processing capabilities of the user device 102, resolution capabilities of one or more cameras of the user device 102, frame rate capabilities of one or more cameras of the user device 102, optical and/or digital zoom capabilities of one or more cameras of the user device 102, audio capture capabilities (e.g., sampling rate, effective range, frequency range, etc.) of one or more audio devices (e.g., microphones) of the user device 102, networking capabilities (e.g., connection speed, latency, transmission speeds) of the user device 102 and/or a current or expected network connection of the user device 102, available audio and/or video codecs, combinations thereof, or the like).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system in which, based on a capability of a device, generates for display a user interface preventing selection of an option indicating one or more entities should not be occluded by an XR portion configured to be inserted in a portion of one or more frames corresponding to a physical background of an environment of a user associated with a device (as taught in Carteri in view of Baron), with determining a device’s capability by determining how quickly content is downloaded to the device (as taught in Zavesky), the motivation being to optimize experience for a user by defining the limitations of the device and adjusting behavior accordingly to the device (see Zavesky Paragraphs [0002] and [0005]).
Regarding Claim 12, Carteri in view of Baron and Zavesky teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining a battery life of the device (see Zavesky Paragraph [0069], In various example embodiments, the device capabilities can represent battery life of the user device 102).
Regarding Claim 15, Carteri in view of Baron and Zavesky teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining whether a codec is supported by the device (see Zavesky Paragraph [0069], In various example embodiments, the device capabilities can represent battery life of the user device 102, memory capacity of the user device 102, processing speed and/or other processing capabilities of the user device 102, resolution capabilities of one or more cameras of the user device 102, frame rate capabilities of one or more cameras of the user device 102, optical and/or digital zoom capabilities of one or more cameras of the user device 102, audio capture capabilities (e.g., sampling rate, effective range, frequency range, etc.) of one or more audio devices (e.g., microphones) of the user device 102, networking capabilities (e.g., connection speed, latency, transmission speeds) of the user device 102 and/or a current or expected network connection of the user device 102, available audio and/or video codecs, combinations thereof, or the like).
Regarding Claim 16, Carteri in view of Baron and Zavesky teaches
The system of claim 1, wherein the control circuitry is configured to determine the capability of the device by determining whether a format is supported by the device (see Zavesky Paragraph [0069], As such, the device capabilities 120 can represent (or can be analyzed to determine) various aspects of the project 136 that is to be created such as, for example, video resolution, video frame rates, video file size, video length, video color depth, video format, combinations thereof, or the like; audio quality, audio file size, audio sampling rate, audio frequency range, audio format, combinations thereof, or the like; project file size, length, format, combinations thereof, or the like; and/or other aspects of the project 136 that is to be created).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Desai et al. (U.S. Pub. No. 2023/0262101, hereinafter “Desai”) in view of Oz et al. (U.S. Pub. No. 2022/0286657, hereinafter “Oz”) and Baron, JR. et al. (U.S. Pub. No. 2012/0050323, hereinafter “Baron”).
Regarding Claim 20, Desai teaches
A system (see Desai Abstract, system) comprising:
a first user device associated with a first user, wherein a server is configured to establish a video communication session over a network, and wherein the video communication session enables the first user device to present a depiction of a second user, associated with a second user device, and an environment surrounding the second user, and the video communication session enables the second user device to present a depiction of the first user and an environment surrounding the first user (see Desai Paragraph [0050], In this illustrative embodiment, each conferencing system terminal device 101,102,103,104 is also in communication with a video conferencing system server complex 106 across the network 105. As will be described in more detail with reference to FIG. 8 below, in one or more embodiments video conferencing system server complex 106 includes components such as a web server, a database server, an audio server, and optionally a video server (the video server may be omitted for audio only conferencing systems) that are operable to facilitate videoconferences between the various conferencing system terminal devices 101,102,103,104 of the videoconference system, and Paragraph [0053], In one or more embodiments, the video conferencing system server complex 106 functions as an intermediary device to facilitate sharing of audio and/or video content and/or data between the various conferencing system terminal devices 101,102,103,104. For example, as can be seen on the displays of conferencing system terminal devices 101,102, in this example each participant 107,108,109,110 can see each other participant engaged in the videoconference), and wherein a portion of the environment surrounding the first user comprises an entity other than the first user (see Desai Figure 6, user is holding object 603 that the camera is capturing that is relevant to the discussion); and
control circuitry (see Desai Paragraph [0111], The operating system environment can be configured as executable code operating on one or more processors or control circuits of the conferencing system terminal device 101)configured to:
determine that an extended reality (XR) portion should be included in a video feed of the video communication session, wherein the video feed comprises the depiction of the first user (see Desai Paragraph [0078], At step 502, the method 500 determines that a videoconference participant is actively engaged in a videoconference with their outgoing video feed turned ON. At step 502, the conferencing system terminal device engaged in the videoconference includes one or more sensors and one or more processors. The one or more processors apply virtual background indicia defining a virtual background in the images behind depictions of the subject. In one or more embodiments, the virtual background applied at step 502 initially blurs everything but the head and torso of the videoconference participant),
detect the entity in the video feed (see Desai Figure 5, detect object in hand, item 510, and Paragraph [0080], In another embodiment, the one or more sensors detect the initiation of the demonstration operation at step 502 by detecting a hand of a participant holding an object 510 moving into a field of view of an image capture device of their conferencing system terminal device. In still other embodiments, the one or more sensors detect the initiation of the demonstration operation at step 502 by detecting a hand of a participant picking up 511 an object and moving the object within the field of view of an image capture device of their conferencing system terminal device);
generate for display a user interface element, at which user input specifying that the entity should not be occluded by the XR portion in the video feed is received, the user interface element including a plurality of options comprising (see Desai Paragraph [0085], where the conferencing system terminal device includes a display, one or more processors of the conferencing system terminal device may present an unblurred preview mode on the display that allows a user selection of the demonstration object. In one or more embodiments, this preview mode is presented prior to causing the virtual background indicia defining the virtual background to appear behind the depictions of the demonstration object of the demonstration operation):
a third option to cause the video feed to comprise the XR portion, the depiction of the first user, and a depiction of one or more objects detected in the environment surrounding the first user (see Desai Paragraph [0089], In one or more embodiments, in response to the one or more sensors detecting initiation of a demonstration operation by the subject at step 503, at step 504 the one or more processors cause the virtual background indicia to appear behind depictions of a demonstration object of the demonstration operation such that the demonstration object is visible against the virtual background indicia in the images transmitted to the at least one remote electronic device engaged in the videoconference. This results in the virtual background indicia obscuring portions of the images depicting objects other than the subject and the demonstration object, and Figure 6, in which object and user are shown in video stream, but remaining area in video is a virtual background); and
receive the user input specifying that the entity should not be occluded by the XR portion in the video feed (see Desai Paragraph [0085], where the conferencing system terminal device includes a display, one or more processors of the conferencing system terminal device may present an unblurred preview mode on the display that allows a user selection of the demonstration object. In one or more embodiments, this preview mode is presented prior to causing the virtual background indicia defining the virtual background to appear behind the depictions of the demonstration object of the demonstration operation);
modify, based on the user input, the video feed to exclude the XR portion from a portion of the video feed depicting the detected entity, wherein the XR portion is included at a different portion of the video feed, and the depiction of the first user is included in the video feed (see Desai Paragraph [0089] In one or more embodiments, in response to the one or more sensors detecting initiation of a demonstration operation by the subject at step 503, at step 504 the one or more processors cause the virtual background indicia to appear behind depictions of a demonstration object of the demonstration operation such that the demonstration object is visible against the virtual background indicia in the images transmitted to the at least one remote electronic device engaged in the videoconference. This results in the virtual background indicia obscuring portions of the images depicting objects other than the subject and the demonstration object); and
transmit the modified video feed to the second user device via the server (see Desai Paragraph [0089] In one or more embodiments, in response to the one or more sensors detecting initiation of a demonstration operation by the subject at step 503, at step 504 the one or more processors cause the virtual background indicia to appear behind depictions of a demonstration object of the demonstration operation such that the demonstration object is visible against the virtual background indicia in the images transmitted to the at least one remote electronic device engaged in the videoconference. This results in the virtual background indicia obscuring portions of the images depicting objects other than the subject and the demonstration object).
Desai does not expressively teach
and the XR portion is configured to occlude the entity in the video feed;
a first option to cause the video feed to comprise the XR portion surrounding the depiction of the first user and occluding any entities in a background surrounding the first user;
a second option to cause the video feed to comprise the XR portion, the depiction of the first user, and a depiction of one or more additional users detected in the environment surrounding the first user;
a fourth option to cause the video feed to comprise no XR portion surrounding the depiction of the first user;
However, Oz teaches
and the XR portion is configured to occlude the entity in the video feed (see Oz Paragraph [0296], If the method detects more than one person in the image, it may ask the user whether to include that person or people in the foreground and in the virtual 3D video conference environment or whether to segment them out of the image and outside of the virtual 3D video conference environment);
a first option to cause the video feed to comprise the XR portion surrounding the depiction of the first user and occluding any entities in a background surrounding the first user (see Oz Paragraph [0296], If the method detects more than one person in the image, it may ask the user whether to include that person or people in the foreground and in the virtual 3D video conference environment or whether to segment them out of the image and outside of the virtual 3D video conference environment, in which would then occlude the entities surrounding first user);
a second option to cause the video feed to comprise the XR portion, the depiction of the first user, and a depiction of one or more additional users detected in the environment surrounding the first user (see Oz Paragraph [0296], If the method detects more than one person in the image, it may ask the user whether to include that person or people in the foreground and in the virtual 3D video conference environment or whether to segment them out of the image and outside of the virtual 3D video conference environment, in which the first user could choose to include other users);
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system of video conferencing in which a first user’s video depicts an entity other than the first user, applying a virtual background to the first user’s video, detecting the entity in the video, in which the virtual background does not occlude the entity per user input or automation, modifying the video stream to output and transmit the first user’s video comprising a virtual background covering the video stream, other than the entity and the first user (as taught in Desai), with occluding entities, other than a first user, in a video using an XR portion, or not occluding a first user and other users in a video stream comprising an XR portion, determined by first user input (as taught in Oz), the motivation being to provide an improved visual interaction in a video conference that allows a user to select what appears in the video stream, thus reducing distractions or interruptions (see Oz Paragraphs [0131], [0296] and [0514]).
Desai in view of Oz does not expressively teach
a fourth option to cause the video feed to comprise no XR portion surrounding the depiction of the first user;
However, Baron teaches
a fourth option to cause the video feed to comprise no XR portion surrounding the depiction of the first user (see Baron Paragraph [0031], After selecting a virtual background to be utilized for a third party 14, computer system 20 can provide the virtual background for review by the user 12 before it is utilized during the videoconference. In this manner, the user 12 is aware of the virtual background that will be utilized during the video conference, and will have an opportunity to prevent an undesired virtual background from being used, in which, if denied, the user could use no virtual background or a different selection);
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system of video conferencing in which a first user’s video depicts an entity other than the first user, applying a virtual background to the first user’s video, detecting the entity in the video, in which the virtual background does not occlude the entity per user input or automation, modifying the video stream to output and transmit the first user’s video comprising a virtual background covering the video stream, other than the entity and the first user (as taught in Desai), with occluding entities, other than a first user, in a video using an XR portion, or not occluding a first user and other users in a video stream comprising an XR portion, determined by first user input (as taught in Oz), the motivation being to provide an improved visual interaction in a video conference that allows a user to select what appears in the video stream, thus reducing distractions or interruptions (see Oz Paragraphs [0131], [0296] and [0514]).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system of video conferencing in which a first user’s video depicts an entity other than the first user, applying a virtual background to the first user’s video that occludes the entity, detecting the entity in the video, in which the user selects to; include an XR portion that does not occlude the entity and the user, occlude entities other than the first user with an XR portion, or not occlude a first user and other users in a video stream with an XR portion, in which the user selects the option in which the entity and the first user should not be occluded by the XR portion of the video stream, thus modifying the video stream to output and transmit the first user’s video accordingly (as taught in Desai in view of Oz), with providing an option to a user to not include an XR portion surrounding the user in a video stream (as taught in Baron), the motivation being to provide an improved user experience in a video conference that allows a user to select what appears in the video stream, thus reducing distractions and allowing users to adapt their video stream in the context of the meeting (see Baron Abstract and Paragraphs [0002] and [0031]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARISSA A JONES whose telephone number is (703)756-1677. The examiner can normally be reached Telework M-F 6:30 AM - 4:00 PM CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 5712727503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARISSA A JONES/Examiner, Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691
Prosecution Insights