DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 8-11, 13 and 17-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klasson et al. (US 2025/0126227), and further in view of Wilson et al. (US 2017/0201722, hereinafter Wilson).
Regarding claim 1, Klasson teaches a computer-implemented method, comprising:
determining a visual cue (analyzing the physical environment of the local video conference participant using object recognition to determine visual cues that are used to create the perception of depth for the 2D video) associated with a real viewing environment physically occupied by a participant (participant 104 occupying the local environment 118, fig. 2B; [0043]: the local video conference participant 104 stands before the two-dimensional video display 108 in local environment 118. The local environment 118 is the physical environment in which the local video conference participant 104 is present. The local video conference participant 104 can move around the local environment 118) within the real viewing environment ([0024]: he present technology provides a three-dimensional video conference experience using readily available cameras and two-dimensional video displays by inserting the two-dimensional video of a remote video conference participant into a three-dimensional environment, and by using visual cues to create the perception of depth for the two-dimensional video of a remote video conference participant; [0029]-[0030]: Some cues for creating the perception of three-dimensions include: “Monocular motion parallax” refers to the visual phenomenon where objects at different distances appear to move at different rates when observed through a single eye. It is based on the concept that as a viewer moves, the relative positions of objects in the scene change due to the difference in their distances from the viewer. The closer objects appear to move faster in the opposite direction of the viewer's movement, while objects farther away appear to move slower or in the same direction as the viewer's movement. This perceptual cue is often utilized in computer vision systems and depth perception analysis; [0035]: When an object occludes another, it serves as a visual cue for depth perception, signaling that the occluded object is located behind the occluding object. The occluded object may only be partially visible or completely hidden, depending on the extent of the occlusion. Occlusion is a fundamental depth cue that our visual system utilizes to understand the relative positions and distances of objects in our environment. It helps us to perceive the spatial arrangement of objects in three-dimensional space, enabling us to infer which objects are in the foreground and which are in the background. In computer graphics and computer vision, occlusion plays a crucial role in rendering realistic and believable scenes. Algorithms are used to simulate occlusion effects, allowing virtual objects to appear realistically in front or behind other objects based on their relative positions in the virtual 3D space. Overall, occlusion is an important visual cue that guides our perception of depth, enhances object recognition, and contributes to our understanding of the spatial relationships between objects; [0066]: the three-dimensional environment includes a least one animated element. The animated element can provide further cues of a three-dimensional environment by utilizing motion to demonstrate monocular motion parallax and other effects. The three-dimensional environment can appear as a live video. In some embodiments, the three-dimensional environment can be constructed from a plurality of layers where some layers include static objects, and other layers include the two-dimensional video of a remote video conference participant, animated scenes, etc.; [0078]: the video conferencing service 310 can use object recognition technologies and other artificial intelligence image analysis tools to identify objects such as walls, desks, lighting sources, etc.; [0092]: In some embodiments, both a remote video conference participant and a local video conference participant can appear in the same environment. In such embodiments, the local video conference participant can create a three-dimensional environment based on an analysis of their local environment, similar to that described above for a remote video conference participant in their physical environment);
rendering, based on the visual cue ([0024]: The present technology provides a three-dimensional video conference experience using readily available cameras and two-dimensional video displays by inserting the two-dimensional video of a remote video conference participant into a three-dimensional environment, and by using visual cues to create the perception of depth for the two-dimensional video of a remote video conference participant), a virtual scene depicting an extension of the real viewing environment that includes the visual cue (remote video conference participant appearing in the 3D environment that the local video conference participant has constructed is functionally analogous to rendering a virtual scene depicting an extension of the real viewing environment that includes the visual cue; [0092]: In some embodiments, both a remote video conference participant and a local video conference participant can appear in the same environment. In such embodiments, the local video conference participant can create a three-dimensional environment based on an analysis of their local environment, similar to that described above for a remote video conference participant in their physical environment. The local video conference participant can then appear to the remote video conference participant in either two-dimensional video with their real environment or as two-dimensional video in the three-dimensional environment the local video conference participant has constructed. At the same time, the remote video conference participant can appear in the three-dimensional environment the local video conference participant has constructed); and
displaying the rendered virtual scene on a display device (fig. 2A-B: two-dimensional video display 108) installed at a fixed location in the real viewing environment ([0054]: FIG. 2B illustrates a picture of the three-dimensional environment displayed on the two-dimensional video display from a perspective taken from inside the local environment according to some aspects of the present technology; [0055]: IG. 2B illustrates the local video conference participant 104 in the local environment 118 looking at the two-dimensional video display 108 displaying the three-dimensional environment with two-dimensional video of the remote video conference participant within it).
Klasson does not explicitly teach the virtual scene is rendered depicting an extension of at least one object or structure physically present in the real environment into the virtual scene, wherein the extension depicted in the virtual scene is aligned with and a virtual continuation of the at least one object or structure.
In a similar field of endeavor, Wilson teaches teach the virtual scene (fig. 2 depicts a tele-immersive experience that is provided to the participants of a tele-immersive virtual conference; [0040]: FIG. 2 depicts the experience of the first participant 202 from the perspective of first participant's local set-up. In that setting, the first participant 202 is standing in a real space 204 and looking at a virtual space 206 created by the environment 100; [0042]: the virtual space 206 seen by the first participant 202 may differ from the virtual space seen by the second participant in one or more respects. For example, as noted above, the first participant can opt to omit his own virtual image 208 from the virtual space 206; likewise, the second participant can opt to omit his own virtual image 210 from his virtual space; [0078]: The participant P1 will perceive a composite virtual scene 326 upon viewing the mirror functionality) is rendered depicting an extension of at least one object or structure (physical table 212, fig. 2) physically present in the real environment into the virtual scene (as shown in fig. 2, a virtual table 218 in virtual space 206 of the environment 100 is a virtual counterpart of the physical table 212 depicted in the environment 100 and can be seen as an extension of the physical table 212 by the first participant 202), wherein the extension depicted in the virtual scene is aligned with and a virtual continuation of the at least one object or structure (as shown in fig. 2, the virtual table 218 displayed in the virtual space 206 is aligned with the physical table 212 and also displayed as a virtual continuation of the physical table 212). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Wilson’s knowledge of rendering a virtual scene displaying a virtual table as an extension of the physical table in the local environment and modify the process of Klasson because such a virtual space may offer an enhanced feeling of immersion to participants of a tele-immersive collaboration system (fig. 2, [0003] and [0025]).
Regarding claim 2, the combination of Klasson and Wilson teaches the computer-implemented method of claim 1, further comprising adapting the real viewing environment by incorporating at least one lighting contribution associated with an additional participant within a second real viewing environment (Klasson - [0048]: Another technique that can help give the three-dimensional environment 102 a feel or more realistic depth when displayed on the two-dimensional video display 108 is to match or adjust the lighting of the three-dimensional environment 102 to the lighting of the video of the remote video conference participant; Klasson - [0077]: if the remote video conference participant has access to an ambient light sensor, a lidar or radar device, or an ultrasound emitting and receiving device, these devices can be used to capture additional data about the physical environment. While these devices might seem specialized, many of these sensors are available on some mobile devices and laptops. Such sensors can be used to measure the light level and light temperature in the physical environment and to create a 3D point cloud of the physical environment which allows for accurate measurements of the physical environment; Klasson - [0078]: The video conferencing service 310 can receive the images, lighting attributes, and depth attributes of the physical environment from the remote video conference participant device 308 and can analyze this data to recognize objects in the physical environment. In some embodiments, the video conferencing service 310 can use object recognition technologies and other artificial intelligence image analysis tools to identify objects such as walls, desks, lighting sources, etc.; Klasson - [0082]: the video conferencing service 310 can place light sources such as windows and lights in appropriate locations, draw shadows of the remote video conference participant and objects that are rendered in the environment, etc. The light sources and shadows can be dynamic as well so that they can be adjusted; Klasson - [0084]: video conferencing service 310 illustrated in FIG. 3 may analyze the two-dimensional video of the remote video conference participant for at least one attribute. The at least one attribute can pertain to a direction or intensity of a light source, or color temperature of the light on the remote video conference participant, etc. illuminating the remote video conference participant; Klasson - [0085]: the method includes adjusting the three-dimensional environment based on the at least one attribute at block 508. For example, the video conferencing service 310 illustrated in FIG. 3 may adjust the three-dimensional environment based on the at least one attribute. The adjusting the three-dimensional environment can include adjusting the lighting of the three-dimensional environment to originate from a similar direction and adjusting the three-dimensional environment for a light saturation that appears natural for the intensity of the light source. For example, a light or window can be dynamically adjusted to match the lighting on the remote video conference participant. In another example, shadows can be rendered to make it appear as if the remote video conference participant is casting a natural-looking shadow based on the lighting in the three-dimensional environment. In another example, the video conferencing service 310 can detect that a light that was present in the images received during the set of the three-dimensional environment is on or off and the video conferencing service 310 can also turn that light on or off. Additionally, if the light is a desk light and it is no longer present, the video conferencing service 310 can dynamically remove the light from the three-dimensional environment; Klasson - [0088]: The video conferencing service 310 can determine that the remote video conference participant is in an unknown environment and can dynamically select the best matching generic environment and can continuously adjust the environment throughout the video conference to improve the lighting characteristics or add light sources as they are discovered; Klasson - [0091]: One of the remote video conference participants can select a virtual environment so that all remote video conference participants can appear together in the same space. While the three-dimensional environment probably won't match either user's physical environment, the three-dimensional environment can be configured with lighting sources that mimic each remote video conference participant's physical environment lighting conditions. For example, the amount of light, color temperature of the light, shadows, etc. can be adjusted for each participant individually; Klasson - [0092]: At the same time, the remote video conference participant can appear in the three-dimensional environment the local video conference participant has constructed. Once again, aspects of the three-dimensional environment can be automatically varied by the video conferencing service 310 to adjust the amount of light, color temperature of the light, shadows, etc. for each participant individually).
Regarding claim 3, the combination of Klasson and Wilson teaches the computer-implemented method of claim 2, wherein the real viewing environment is adapted by controlling lighting using the display device (Klasson - [0048]: Another technique that can help give the three-dimensional environment 102 a feel or more realistic depth when displayed on the two-dimensional video display 108 is to match or adjust the lighting of the three-dimensional environment 102 to the lighting of the video of the remote video conference participant; Klasson - [0085]: the method includes adjusting the three-dimensional environment based on the at least one attribute at block 508. For example, the video conferencing service 310 illustrated in FIG. 3 may adjust the three-dimensional environment based on the at least one attribute. The adjusting the three-dimensional environment can include adjusting the lighting of the three-dimensional environment to originate from a similar direction and adjusting the three-dimensional environment for a light saturation that appears natural for the intensity of the light source. For example, a light or window can be dynamically adjusted to match the lighting on the remote video conference participant. In another example, shadows can be rendered to make it appear as if the remote video conference participant is casting a natural-looking shadow based on the lighting in the three-dimensional environment. In another example, the video conferencing service 310 can detect that a light that was present in the images received during the set of the three-dimensional environment is on or off and the video conferencing service 310 can also turn that light on or off. Additionally, if the light is a desk light and it is no longer present, the video conferencing service 310 can dynamically remove the light from the three-dimensional environment; Klasson - [0088]: The video conferencing service 310 can determine that the remote video conference participant is in an unknown environment and can dynamically select the best matching generic environment and can continuously adjust the environment throughout the video conference to improve the lighting characteristics or add light sources as they are discovered; Klasson - [0091]: One of the remote video conference participants can select a virtual environment so that all remote video conference participants can appear together in the same space. While the three-dimensional environment probably won't match either user's physical environment, the three-dimensional environment can be configured with lighting sources that mimic each remote video conference participant's physical environment lighting conditions. For example, the amount of light, color temperature of the light, shadows, etc. can be adjusted for each participant individually; Klasson - [0092]: At the same time, the remote video conference participant can appear in the three-dimensional environment the local video conference participant has constructed. Once again, aspects of the three-dimensional environment can be automatically varied by the video conferencing service 310 to adjust the amount of light, color temperature of the light, shadows, etc. for each participant individually).
Regarding claim 4, the combination of Klasson and Wilson teaches the computer-implemented method of claim 2, wherein the real viewing environment is adapted by controlling lighting using an additional device (Klasson - [0048]: Another technique that can help give the three-dimensional environment 102 a feel or more realistic depth when displayed on the two-dimensional video display 108 is to match or adjust the lighting of the three-dimensional environment 102 to the lighting of the video of the remote video conference participant; Klasson - [0085]: the method includes adjusting the three-dimensional environment based on the at least one attribute at block 508. For example, the video conferencing service 310 illustrated in FIG. 3 may adjust the three-dimensional environment based on the at least one attribute. The adjusting the three-dimensional environment can include adjusting the lighting of the three-dimensional environment to originate from a similar direction and adjusting the three-dimensional environment for a light saturation that appears natural for the intensity of the light source. For example, a light or window can be dynamically adjusted to match the lighting on the remote video conference participant. In another example, shadows can be rendered to make it appear as if the remote video conference participant is casting a natural-looking shadow based on the lighting in the three-dimensional environment. In another example, the video conferencing service 310 can detect that a light that was present in the images received during the set of the three-dimensional environment is on or off and the video conferencing service 310 can also turn that light on or off. Additionally, if the light is a desk light and it is no longer present, the video conferencing service 310 can dynamically remove the light from the three-dimensional environment; Klasson - [0088]: The video conferencing service 310 can determine that the remote video conference participant is in an unknown environment and can dynamically select the best matching generic environment and can continuously adjust the environment throughout the video conference to improve the lighting characteristics or add light sources as they are discovered; Klasson - [0091]: One of the remote video conference participants can select a virtual environment so that all remote video conference participants can appear together in the same space. While the three-dimensional environment probably won't match either user's physical environment, the three-dimensional environment can be configured with lighting sources that mimic each remote video conference participant's physical environment lighting conditions. For example, the amount of light, color temperature of the light, shadows, etc. can be adjusted for each participant individually; Klasson - [0092]: At the same time, the remote video conference participant can appear in the three-dimensional environment the local video conference participant has constructed. Once again, aspects of the three-dimensional environment can be automatically varied by the video conferencing service 310 to adjust the amount of light, color temperature of the light, shadows, etc. for each participant individually).
Regarding claim 8, the combination of Klasson and Wilson teaches the computer-implemented method of claim 1, further comprising receiving head tracking data for the participant (Klasson - [0020]: tracking a position of the local video conference participant in a physical environment before the two-dimensional video display; Klasson - [0044]: The present technology can use face detection and tracking to determine the head position of the local video conference participant 104; Klasson - [0069]: the method includes tracking the position of the local video conference participant in a physical environment before the two-dimensional video display at block 412. For example, the video conferencing service 310 illustrated in FIG. 3 may track the position of the local video conference participant in a physical environment before the two-dimensional video display. The tracking the position of the local video conference participant includes tracking the face or head of the human local video conference participant as the human local video conference participant moves in the physical three-dimensional environment … the tracking can be performed by the video capture device 106; Wilson - [0054]: the environment 100 can track the movement of each participant's hands, and/or head, and/or eyes, and/or entire body; Wilson – [0093]: the tracking module 504 can use any head movement technology to track the movement of the participant's head); and in response to the head tracking data, rendering the virtual scene from a viewpoint corresponding to the head tracking data (Klasson - [0020]: tracking a position of the local video conference participant in a physical environment before the two-dimensional video display, and translating the three-dimensional environment in response to a change in the position of the local video conference participant device in the physical environment; Klasson - [0056]: A video capture device 106 can capture video of the local video conference participant 104 and analyze the video to track the video capture device 106 throughout the local environment 118 in order to adjust the perspective and point of view of the three-dimensional environment so that the local video conference participant 104 has the impression that they are looking through a window at a live three-dimensional environment; Klasson - [0067]: he video capture device 106 might have algorithms to perform the identifying the face or head and for tracking the local video conference participant 104 and can provide these processed results to the video conferencing service 310).
Regarding claim 9, the combination of Klasson and Wilson teaches the computer-implemented method of claim 8, further comprising transmitting the head tracking data and data associated with the real viewing environment to a display system in an additional real viewing environment (Klasson - as shown in fig. 3, the remote video conference participant and local video conference participant can transmit and receive data from each other using the network 302; Klasson - [0057]: the video capture device 106 can also be used to transmit video of the local video conference participant 104 to a video conferencing service for viewing by the remote video conference participant; Klasson - [0062]: The video conferencing service 310 can transmit the two-dimensional video of the remote video conference participant over the network 302 to video conferencing equipment of the side of the local video conference participant 104; Klasson - [0067]: According to some examples, the method includes identifying a face or head of a human local video conference participant by the local video conference participant device at block 408. For example, the video conferencing service 310 illustrated in FIG. 3 may identify a face or head of a human local video conference participant by the local video conference participant device. The method further includes displaying the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display at block 410. For example, the two-dimensional video display 108 illustrated in FIG. 1B may display the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display. Collectively these functions can be performed by a combination of the video capture device 106 and video conferencing service 310 wherein the video capture device 106 can capture video of the local video conference participant 104 and send it back to the video conferencing service 310 which can process the video to determine the point of view of the local video conference participant 104 and adjust the rendering of the three-dimensional environment accordingly. In some embodiments, the video capture device 106 might have algorithms to perform the identifying the face or head and for tracking the local video conference participant 104 and can provide these processed results to the video conferencing service 310. Alternatively, such as when the three-dimensional environment is rendered by the two-dimensional video display 108, the two-dimensional video display 108 can perform the functions of deriving the point of view of the local video conference participant 104 (or may receive such information from the video capture device 106) and adjust the rendering of the three-dimensional environment accordingly).
Regarding claim 10, the combination of Klasson and Wilson teaches the computer-implemented method of claim 8, further comprising receiving additional head tracking data corresponding to a view location of an additional participant from a display system in an additional real viewing environment (Klasson - as shown in fig. 3, the remote video conference participant and local video conference participant can transmit and receive data from each other using the network 302; Klasson - [0057]: the video capture device 106 can also be used to transmit video of the local video conference participant 104 to a video conferencing service for viewing by the remote video conference participant; Klasson - [0062]: The video conferencing service 310 can transmit the two-dimensional video of the remote video conference participant over the network 302 to video conferencing equipment of the side of the local video conference participant 104; Klasson - [0067]: According to some examples, the method includes identifying a face or head of a human local video conference participant by the local video conference participant device at block 408. For example, the video conferencing service 310 illustrated in FIG. 3 may identify a face or head of a human local video conference participant by the local video conference participant device. The method further includes displaying the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display at block 410. For example, the two-dimensional video display 108 illustrated in FIG. 1B may display the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display. Collectively these functions can be performed by a combination of the video capture device 106 and video conferencing service 310 wherein the video capture device 106 can capture video of the local video conference participant 104 and send it back to the video conferencing service 310 which can process the video to determine the point of view of the local video conference participant 104 and adjust the rendering of the three-dimensional environment accordingly. In some embodiments, the video capture device 106 might have algorithms to perform the identifying the face or head and for tracking the local video conference participant 104 and can provide these processed results to the video conferencing service 310. Alternatively, such as when the three-dimensional environment is rendered by the two-dimensional video display 108, the two-dimensional video display 108 can perform the functions of deriving the point of view of the local video conference participant 104 (or may receive such information from the video capture device 106) and adjust the rendering of the three-dimensional environment accordingly).
Regarding claim 11, the combination of Klasson and Wilson teaches the computer-implemented method of claim 1, wherein the visual cue reinforces connections between the virtual scene and at least one object or structure physically present in the real viewing environment (Wilson – fig. 2 shows the continuation of the physical table 212 as the virtual table 218 in the virtual space that appears to be connected as one single table to the participant in the local environment 100; Klasson – [0024]: The present technology provides a three-dimensional video conference experience using readily available cameras and two-dimensional video displays by inserting the two-dimensional video of a remote video conference participant into a three-dimensional environment, and by using visual cues to create the perception of depth for the two-dimensional video of a remote video conference participant; Wilson – claim 26: the image capture functionality is configured to provide depth information for use in constructing a depth image of the local participant) and comprises at least one of similarity, proximity, closure, or continuation (Klasson – [0034]: relative size is a visual cue used to perceive the size and distance of objects in relation to one another; Klasson – [0035]: occlusion is an important visual cue that guides our perception of depth, enhances object recognition, and contributes to our understanding of the spatial relationships between objects; Wilson - Wilson – fig. 2 shows the continuation of the physical table 212 as the virtual table 218 in the virtual space).
Regarding claim 13, the combination of Klasson and Wilson teaches the computer-implemented method of claim 1, wherein at least one of the steps of determining or rendering is performed on a server or in a data center and the virtual scene is streamed to a user device (Klasson - [0065]: the video conferencing service 310 illustrated in FIG. 3 may render a three-dimensional environment using a three-dimensional rendering engine such as UNITY or UNREAL; Klasson - [0067]: the method includes identifying a face or head of a human local video conference participant by the local video conference participant device at block 408. For example, the video conferencing service 310 illustrated in FIG. 3 may identify a face or head of a human local video conference participant by the local video conference participant device. The method further includes displaying the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display at block 410. For example, the two-dimensional video display 108 illustrated in FIG. 1B may display the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display. Collectively these functions can be performed by a combination of the video capture device 106 and video conferencing service 310 wherein the video capture device 106 can capture video of the local video conference participant 104 and send it back to the video conferencing service 310 which can process the video to determine the point of view of the local video conference participant 104 and adjust the rendering of the three-dimensional environment accordingly; Klasson - [0076]: the video conferencing service 310 illustrated in FIG. 33, can analyze the physical environment of the remote video conference participant; Klasson - [0110]: Any of the steps, operations, functions, or processes described herein may be performed or implemented by a combination of hardware and software services or services, alone or in combination with other devices. In some embodiments, a service can be software that resides in memory of a client device and/or one or more servers of a content management system and perform one or more functions when a processor executes the software associated with the service. In some embodiments, a service is a program, or a collection of programs that carry out a specific function. In some embodiments, a service can be considered a server).
Claims 17, 18, 19 and 20 are similar in scope to claims 1, 2, 1 and 2, respectively, and therefore the examiner provides similar rationale to reject these claims. Moreover, Klasson teaches a system (Klasson - system 700, fig. 7) comprising a display device (Klasson - two-dimensional video display 108, figs. 1B, 2A, 2B; output device 772, fig. 7) installed at a fixed location in a real viewing environment (Klasson - [0047]: mount the two-dimensional video display 108 in a recessed frame or in a wall cavity), and a processor coupled to the display (Klasson - processor 704, fig. 7 and [0104-0105]). Klasson further teaches a non-transitory computer readable medium (Klasson - [0108], [0110]).
Regarding claim 21, the combination of Klasson and Wilson teaches the computer-implemented method of claim 1, wherein the extension depicted in the virtual scene comprises a portion of the at least one object or structure (Wilson - fig. 2 shows an extension of the physical table 212 as the virtual table 218 in the virtual space 206).
Regarding claim 22, the combination of Klasson and Wilson does not explicitly teach the computer-implemented method of claim 1, wherein the extension depicted in the virtual scene comprises a continuation of at least one of a real wall-wall intersection, floor-wall intersection, or ceiling-wall intersection.
However, the combination of Klasson and Wilson teaches the extension depicted in the virtual scene comprises a continuation of a side of the physical table (Wilson - as shown in fig. 2, a virtual table 218 in virtual space 206 of the environment 100 is a virtual counterpart of the physical table 212 depicted in the environment 100 and can be seen as an extension and continuation of the shorter side of the physical table 212). It would have been prima facie obvious for the extension to depict a continuation of at least one of a real wall-wall intersection, floor-wall intersection, or ceiling-wall intersection. Whether the extension depicted in the virtual scene comprises a continuation of the side of the physical table as taught by the combination of Klasson and Wilson or a continuation of at least one of a real wall-wall intersection, floor-wall intersection, or ceiling-wall intersection is solely a matter of aesthetic design choice, and would not be sufficient to distinguish over the prior art. See MPEP 2144.04.
Claim(s) 5-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klasson, in view of Wilson, and further in view of Shim (US 2011/0227922).
Regarding claim 5, the combination of Klasson and Wilson teaches the computer-implemented method of claim 2, further comprising: receiving a representation of the additional participant (Klasson - fig. 4 step 402: receiving a video stream including 2D video of a remote video conference participant); (Klasson - [0026]: extract the video of the participant from their background; [0048]: the lighting of the video of the remote video conference participant; Klasson - [0071]: the video capture device 106 or other sensors (ambient light sensors) can detect the amount of light in the local environment; Klasson - [0077]: the remote video conference participant can operate their remote video conference participant device 308 or other environment-capturing equipment to capture images and depth attributes of their physical environment … if the remote video conference participant has access to an ambient light sensor, a lidar or radar device, or an ultrasound emitting and receiving device, these devices can be used to capture additional data about the physical environment. While these devices might seem specialized, many of these sensors are available on some mobile devices and laptops. Such sensors can be used to measure the light level and light temperature in the physical environment; Klasson - [0078]: The video conferencing service 310 can receive the images, lighting attributes, and depth attributes of the physical environment from the remote video conference participant device 308 and can analyze this data to recognize objects in the physical environment); and controlling lighting within the real viewing environment to simulate the at least one lighting contribution (Klasson - [0085]: the method includes adjusting the three-dimensional environment based on the at least one attribute at block 508. For example, the video conferencing service 310 illustrated in FIG. 3 may adjust the three-dimensional environment based on the at least one attribute. The adjusting the three-dimensional environment can include adjusting the lighting of the three-dimensional environment to originate from a similar direction and adjusting the three-dimensional environment for a light saturation that appears natural for the intensity of the light source. For example, a light or window can be dynamically adjusted to match the lighting on the remote video conference participant. In another example, shadows can be rendered to make it appear as if the remote video conference participant is casting a natural-looking shadow based on the lighting in the three-dimensional environment. In another example, the video conferencing service 310 can detect that a light that was present in the images received during the set of the three-dimensional environment is on or off and the video conferencing service 310 can also turn that light on or off. Additionally, if the light is a desk light and it is no longer present, the video conferencing service 310 can dynamically remove the light from the three-dimensional environment; Klasson - [0088]: The video conferencing service 310 can determine that the remote video conference participant is in an unknown environment and can dynamically select the best matching generic environment and can continuously adjust the environment throughout the video conference to improve the lighting characteristics or add light sources as they are discovered; Klasson - [0091]: One of the remote video conference participants can select a virtual environment so that all remote video conference participants can appear together in the same space. While the three-dimensional environment probably won't match either user's physical environment, the three-dimensional environment can be configured with lighting sources that mimic each remote video conference participant's physical environment lighting conditions. For example, the amount of light, color temperature of the light, shadows, etc. can be adjusted for each participant individually; Klasson - [0092]: At the same time, the remote video conference participant can appear in the three-dimensional environment the local video conference participant has constructed. Once again, aspects of the three-dimensional environment can be automatically varied by the video conferencing service 310 to adjust the amount of light, color temperature of the light, shadows, etc. for each participant individually).
Klasson does not explicitly teach extracting the at least one lighting contribution from the representation.
Shim teaches extracting the at least one lighting contribution from the representation ([0025]: The foregoing and/or other aspects are achieved by providing inverse rendering, by at least one processor, an input image into a specular reflection portion and a diffuse reflection portion, extracting, by the at least one processor, light information from the specular reflection portion and the diffuse reflection portion and shape information including geometric information of the input image, and extracting, by the at least one processor, texture information from the specular reflection portion, the diffuse reflection portion and the shape information; [0036]: The light and texture extracting apparatus 100 to perform rendering may be an apparatus to extract light information and texture information associated with an input image by performing inverse-rendering of the input image and shape information associated with the input image). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Shim’s knowledge of extracting lighting information from an input image as taught and modify the process of Klasson and Wilson because such a process uses the extracted light information to render another image for more realistic rendering (abstract and [0073]).
Regarding claim 6, the combination of Klasson, Wilson and Shim teaches the computer-implemented method of claim 5, wherein the representation is an image and the at least one lighting contribution is extracted using inverse rendering (Klasson - [0025]: The foregoing and/or other aspects are achieved by providing inverse rendering, by at least one processor, an input image into a specular reflection portion and a diffuse reflection portion, extracting, by the at least one processor, light information from the specular reflection portion and the diffuse reflection portion and shape information including geometric information of the input image, and extracting, by the at least one processor, texture information from the specular reflection portion, the diffuse reflection portion and the shape information; Klasson - [0036]: The light and texture extracting apparatus 100 to perform rendering may be an apparatus to extract light information and texture information associated with an input image by performing inverse-rendering of the input image and shape information associated with the input image).
Regarding claim 7, the combination of Klasson and Shim teaches the computer-implemented method of claim 5, further comprising rendering the representation to match the real environment lighting (Klasson - [0048]: match or adjust the lighting of the three-dimensional environment 102 to the lighting of the video of the remote video conference participant; Klasson - [0075]: One method that can be used to make the two-dimensional video of the remote video conference participant look more natural in the virtual background is to place the two-dimensional video of the remote video conference participant in a virtual environment that matches the remote video conference participant's physical environment as much as possible; Klasson - [0085]: a light or window can be dynamically adjusted to match the lighting on the remote video conference participant; Klasson - [0092]: aspects of the three-dimensional environment can be automatically varied by the video conferencing service 310 to adjust the amount of light, color temperature of the light, shadows, etc. for each participant individually).
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klasson, in view of Wilson, and further in view of Kerofsky et al. (US 2016/0255322, hereinafter Kerofsky).
Regarding claim 12, although the combination of Klasson and Wilson teaches tracking a position of the local video conference participant in a physical environment before the two-dimensional video display, and translating the three-dimensional environment in response to a change in the position of the local video conference participant device in the physical environment (Klasson - [0020]), the combination of Klasson and Wilson does not explicitly teach the computer-implemented method of claim 1, wherein the display device is one of a head-tracked stereoscopic display, true light field display, or a motion parallax display.
Kerofsky teaches the display device is one of a head-tracked stereoscopic display, true light field display, or a motion parallax display ([0061]: In smaller displays, the user's position may vary significantly relative to the display, making motion parallax a strong depth cue; [0079]: This method may support intuitive user interaction with static 3D objects and/or with modeled 3D environments on a 2D display via motion parallax-based rendering tied to viewer position estimate). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Kerofsky’s knowledge of using a motion parallax display as taught and modify the process of Klasson and Wilson because such a process supports intuitive user interaction with static 3D objects and/or with modeled 3D environments on a 2D display ([0079]).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klasson, in view of Wilson, and further in view of Bamberger et al. (US 2018/0246329, hereinafter Bamberger).
Regarding claim 14, the combination of Klasson and Wilson teaches the computer-implemented method of claim 1, wherein at least one of the steps of determining, rendering, or displaying is performed (Klasson - [0065]: the video conferencing service 310 illustrated in FIG. 3 may render a three-dimensional environment using a three-dimensional rendering engine such as UNITY or UNREAL; Klasson - [0067]: the method includes identifying a face or head of a human local video conference participant by the local video conference participant device at block 408. For example, the video conferencing service 310 illustrated in FIG. 3 may identify a face or head of a human local video conference participant by the local video conference participant device. The method further includes displaying the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display at block 410. For example, the two-dimensional video display 108 illustrated in FIG. 1B may display the three-dimensional environment relative to a point of view determined from a position of the identified face or head of the human local video conference participant relative to the two-dimensional video display. Collectively these functions can be performed by a combination of the video capture device 106 and video conferencing service 310 wherein the video capture device 106 can capture video of the local video conference participant 104 and send it back to the video conferencing service 310 which can process the video to determine the point of view of the local video conference participant 104 and adjust the rendering of the three-dimensional environment accordingly; Klasson - [0076]: the video conferencing service 310 illustrated in FIG. 33, can analyze the physical environment of the remote video conference participant; Klasson - [0110]: Any of the steps, operations, functions, or processes described herein may be performed or implemented by a combination of hardware and software services or services, alone or in combination with other devices. In some embodiments, a service can be software that resides in memory of a client device and/or one or more servers of a content management system and perform one or more functions when a processor executes the software associated with the service. In some embodiments, a service is a program, or a collection of programs that carry out a specific function. In some embodiments, a service can be considered a server).
The combination of Klasson and Wilson does not explicitly teach the server is implemented in a cloud computing environment.
Bamberger teaches the server is implemented in a cloud computing environment ([0032]: The server system 102 may be a cloud computing environment, according to some example embodiments. The server system 102, and any servers associated with the server system 102, may be associated with a cloud-based application, in one example embodiment). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Bamberger’s knowledge of implementing a server in a cloud computing environment as taught and modify the process of Klasson and Wilson because such a process allows scalability to adjust resources as per need.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klasson, in view of Wilson, and further in view of Tang et al. (US 2023/0169324, hereinafter Tang).
Regarding claim 15, the combination of Klasson and Wilson does not explicitly teach the computer-implemented method of claim 1, wherein the virtual scene is used for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle.
Tang teaches the virtual scene is used for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle ([0001]: a system and method for training a neural network and, more particularly, to a system and method for training a neural network that could be used in a robot controller for identifying a box to be picked up by the robot from a stack of boxes, where the method employs using computer graphics software to generate virtual images of the boxes and automatically labelling the boxes based on their orientation in the virtual images; [0020]: a system and method for training a neural network that could be used for identifying a box to be picked up by a robot from a stack of boxes, where the method employs using computer graphics software to generate virtual images of the boxes and automatically labelling the boxes based on their orientation in the virtual images). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Tang’s knowledge of generating virtual images to train a neural network as taught and modify the process of Klasson and Wilson because such a process to train the neural networks is less expensive and saves time otherwise needed for preparing dataset for training deep learning neural networks ([0023]).
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klasson, in view of Wilson, and further in view of van Welzen et al. (US 2022/0331702, hereinafter Welzen).
Regarding claim 16, the combination of Klasson and Wilson does not explicitly teach the computer-implemented method of claim 1, wherein at least one of the steps of determining, rendering, or displaying is performed on a virtual machine comprising a portion of a graphics processing unit.
Welzen teaches at least one of the steps of determining, rendering, or displaying is performed on a virtual machine comprising a portion of a graphics processing unit ([0046]: The compositing server(s) 106 may use a virtual machine(s) 133 to execute the compositing application 128 as an instance of a compositing application 130. For instance, the virtual machine(s) 133 (including one or more virtual GPUs, one or more discrete GPUs, one or more virtual CPUs, and/or one or more discrete CPUs) may render one or more user interfaces). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Welzen’s knowledge of rendering using a virtual machine as taught and modify the process of Klasson and Wilson because such a process enhances the user experience by providing the user with the ability to modify layouts and adjust the composited content ([0046]).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on the same combination of references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Response to the arguments that Klasson does not teach alignment with an continuation of an object or structure between the real viewing environment and the corresponding extension across the frame boundary into the virtual scene. See page 8 of Applicant’s Remarks filed on 12/16/2025.
Klasson in view of Wilson teaches to render the virtual scene depicting an extension of at least one object or structure physically present in the real environment into the virtual scene, wherein the extension depicted in the virtual scene is aligned with and a virtual continuation of the at least one object or structure.
Especially, Wilson teaches teach the virtual scene (fig. 2 depicts a tele-immersive experience that is provided to the participants of a tele-immersive virtual conference; [0040]: FIG. 2 depicts the experience of the first participant 202 from the perspective of first participant's local set-up. In that setting, the first participant 202 is standing in a real space 204 and looking at a virtual space 206 created by the environment 100; [0042]: the virtual space 206 seen by the first participant 202 may differ from the virtual space seen by the second participant in one or more respects. For example, as noted above, the first participant can opt to omit his own virtual image 208 from the virtual space 206; likewise, the second participant can opt to omit his own virtual image 210 from his virtual space; [0078]: The participant P1 will perceive a composite virtual scene 326 upon viewing the mirror functionality) is rendered depicting an extension of at least one object or structure (physical table 212, fig. 2) physically present in the real environment into the virtual scene (as shown in fig. 2, a virtual table 218 in virtual space 206 of the environment 100 is a virtual counterpart of the physical table 212 depicted in the environment 100 and can be seen as an extension of the physical table 212 by the first participant 202), wherein the extension depicted in the virtual scene is aligned with and a virtual continuation of the at least one object or structure (as shown in fig. 2, the virtual table 218 displayed in the virtual space 206 is aligned with the physical table 212 and also displayed as a virtual continuation of the physical table 212).
Response to the arguments that Klasson does not teach altering the local room illumination using the display or auxiliary lights to stimulate remote participants’ lighting. See page 9 of Applicant’s Remarks filed on 12/16/2025.
Examiner disagrees. Klasson teaches to adapt the real viewing environment (three-dimensional environment 102 is displayed on the two-dimensional video display 108 that is part of the local environment 118) by incorporating at least one lighting contribution associated with an additional participant (remote video conference participant, fig. 2B) within a second real viewing environment (environment of the remote video conference participant; the lighting of the three-dimensional environment 102 displayed on the two-dimensional video display 108 that is a part of the local environment 118 is adjusted or controlled by the lighting of the video of the remote video conference participant, where the attribute such intensity of a light source of the two-dimensional video display 108 is adjusted to control the light of the three-dimensional environment 102 to match the lighting of the remote video conference participant’s environment; Klasson - [0048]: Another technique that can help give the three-dimensional environment 102 a feel or more realistic depth when displayed on the two-dimensional video display 108 is to match or adjust the lighting of the three-dimensional environment 102 to the lighting of the video of the remote video conference participant; Klasson - [0077]: if the remote video conference participant has access to an ambient light sensor, a lidar or radar device, or an ultrasound emitting and receiving device, these devices can be used to capture additional data about the physical environment. While these devices might seem specialized, many of these sensors are available on some mobile devices and laptops. Such sensors can be used to measure the light level and light temperature in the physical environment and to create a 3D point cloud of the physical environment which allows for accurate measurements of the physical environment; Klasson - [0078]: The video conferencing service 310 can receive the images, lighting attributes, and depth attributes of the physical environment from the remote video conference participant device 308 and can analyze this data to recognize objects in the physical environment. In some embodiments, the video conferencing service 310 can use object recognition technologies and other artificial intelligence image analysis tools to identify objects such as walls, desks, lighting sources, etc.; Klasson - [0082]: the video conferencing service 310 can place light sources such as windows and lights in appropriate locations, draw shadows of the remote video conference participant and objects that are rendered in the environment, etc. The light sources and shadows can be dynamic as well so that they can be adjusted; Klasson - [0084]: video conferencing service 310 illustrated in FIG. 3 may analyze the two-dimensional video of the remote video conference participant for at least one attribute. The at least one attribute can pertain to a direction or intensity of a light source, or color temperature of the light on the remote video conference participant, etc. illuminating the remote video conference participant; Klasson - [0085]: the method includes adjusting the three-dimensional environment based on the at least one attribute at block 508. For example, the video conferencing service 310 illustrated in FIG. 3 may adjust the three-dimensional environment based on the at least one attribute. The adjusting the three-dimensional environment can include adjusting the lighting of the three-dimensional environment to originate from a similar direction and adjusting the three-dimensional environment for a light saturation that appears natural for the intensity of the light source. For example, a light or window can be dynamically adjusted to match the lighting on the remote video conference participant. In another example, shadows can be rendered to make it appear as if the remote video conference participant is casting a natural-looking shadow based on the lighting in the three-dimensional environment. In another example, the video conferencing service 310 can detect that a light that was present in the images received during the set of the three-dimensional environment is on or off and the video conferencing service 310 can also turn that light on or off. Additionally, if the light is a desk light and it is no longer present, the video conferencing service 310 can dynamically remove the light from the three-dimensional environment; Klasson - [0088]: The video conferencing service 310 can determine that the remote video conference participant is in an unknown environment and can dynamically select the best matching generic environment and can continuously adjust the environment throughout the video conference to improve the lighting characteristics or add light sources as they are discovered; Klasson - [0091]: One of the remote video conference participants can select a virtual environment so that all remote video conference participants can appear together in the same space. While the three-dimensional environment probably won't match either user's physical environment, the three-dimensional environment can be configured with lighting sources that mimic each remote video conference participant's physical environment lighting conditions. For example, the amount of light, color temperature of the light, shadows, etc. can be adjusted for each participant individually; Klasson - [0092]: At the same time, the remote video conference participant can appear in the three-dimensional environment the local video conference participant has constructed. Once again, aspects of the three-dimensional environment can be automatically varied by the video conferencing service 310 to adjust the amount of light, color temperature of the light, shadows, etc. for each participant individually).
Response to the arguments that Klasson does not teach to control the lighting within the real viewing environment (i.e., physically altering the lighting in the local room) to simulate another participant’s lighting, and further argues that Klasson and Shim do not teach to control lighting within the real viewing environment to simulate the extracted remote lighting contribution. See page 9-10 of Applicant’s Remarks filed on 12/16/2025.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., physically altering the lighting in the local room) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Moreover, Klasson teaches to control lighting within the real viewing environment to simulate the extracted remote lighting contribution (the lighting of the three-dimensional environment 102 displayed on the two-dimensional video display 108 that is a part of the local environment 118 is adjusted or controlled by the lighting of the video of the remote video conference participant, where the attribute such intensity of a light source of the two-dimensional video display 108 is adjusted to control the light of the three-dimensional environment 102 to match the lighting of the remote video conference participant’s environment.
Especially, Klasson teaches receiving a representation of the additional participant (Klasson - fig. 4 step 402: receiving a video stream including 2D video of a remote video conference participant); (Klasson - [0026]: extract the video of the participant from their background; [0048]: the lighting of the video of the remote video conference participant; Klasson - [0071]: the video capture device 106 or other sensors (ambient light sensors) can detect the amount of light in the local environment; Klasson - [0077]: the remote video conference participant can operate their remote video conference participant device 308 or other environment-capturing equipment to capture images and depth attributes of their physical environment … if the remote video conference participant has access to an ambient light sensor, a lidar or radar device, or an ultrasound emitting and receiving device, these devices can be used to capture additional data about the physical environment. While these devices might seem specialized, many of these sensors are available on some mobile devices and laptops. Such sensors can be used to measure the light level and light temperature in the physical environment; Klasson - [0078]: The video conferencing service 310 can receive the images, lighting attributes, and depth attributes of the physical environment from the remote video conference participant device 308 and can analyze this data to recognize objects in the physical environment); and controlling lighting within the real viewing environment to simulate the at least one lighting contribution (Klasson - [0085]: the method includes adjusting the three-dimensional environment based on the at least one attribute at block 508. For example, the video conferencing service 310 illustrated in FIG. 3 may adjust the three-dimensional environment based on the at least one attribute. The adjusting the three-dimensional environment can include adjusting the lighting of the three-dimensional environment to originate from a similar direction and adjusting the three-dimensional environment for a light saturation that appears natural for the intensity of the light source. For example, a light or window can be dynamically adjusted to match the lighting on the remote video conference participant. In another example, shadows can be rendered to make it appear as if the remote video conference participant is casting a natural-looking shadow based on the lighting in the three-dimensional environment. In another example, the video conferencing service 310 can detect that a light that was present in the images received during the set of the three-dimensional environment is on or off and the video conferencing service 310 can also turn that light on or off. Additionally, if the light is a desk light and it is no longer present, the video conferencing service 310 can dynamically remove the light from the three-dimensional environment; Klasson - [0088]: The video conferencing service 310 can determine that the remote video conference participant is in an unknown environment and can dynamically select the best matching generic environment and can continuously adjust the environment throughout the video conference to improve the lighting characteristics or add light sources as they are discovered; Klasson - [0091]: One of the remote video conference participants can select a virtual environment so that all remote video conference participants can appear together in the same space. While the three-dimensional environment probably won't match either user's physical environment, the three-dimensional environment can be configured with lighting sources that mimic each remote video conference participant's physical environment lighting conditions. For example, the amount of light, color temperature of the light, shadows, etc. can be adjusted for each participant individually; Klasson - [0092]: At the same time, the remote video conference participant can appear in the three-dimensional environment the local video conference participant has constructed. Once again, aspects of the three-dimensional environment can be automatically varied by the video conferencing service 310 to adjust the amount of light, color temperature of the light, shadows, etc. for each participant individually).
Shim teaches extracting the at least one lighting contribution from the representation ([0025]: The foregoing and/or other aspects are achieved by providing inverse rendering, by at least one processor, an input image into a specular reflection portion and a diffuse reflection portion, extracting, by the at least one processor, light information from the specular reflection portion and the diffuse reflection portion and shape information including geometric information of the input image, and extracting, by the at least one processor, texture information from the specular reflection portion, the diffuse reflection portion and the shape information; [0036]: The light and texture extracting apparatus 100 to perform rendering may be an apparatus to extract light information and texture information associated with an input image by performing inverse-rendering of the input image and shape information associated with the input image).
Response to the arguments that Klasson does not teach using any cue to reinforce connections between the virtual scene and a specific object or structure physically present in the viewer’s real environment. See page 10-11 of Applicant’s Remarks filed on 12/16/2025.
Klasson in view of Wilson teaches the above limitation. Especially, Wilson teaches the visual cue reinforces connections between the virtual scene and at least one object or structure physically present in the real viewing environment (Wilson – fig. 2 shows the continuation of the physical table 212 as the virtual table 218 in the virtual space that appears to be connected as one single table to the participant in the local environment 100; Klasson – [0024]: The present technology provides a three-dimensional video conference experience using readily available cameras and two-dimensional video displays by inserting the two-dimensional video of a remote video conference participant into a three-dimensional environment, and by using visual cues to create the perception of depth for the two-dimensional video of a remote video conference participant; Wilson – claim 26: the image capture functionality is configured to provide depth information for use in constructing a depth image of the local participant).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JWALANT B AMIN whose telephone number is (571)272-2455. The examiner can normally be reached Monday-Friday 10am - 630pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JWALANT AMIN/Primary Examiner, Art Unit 2612