DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims:
Claims 1 - 4, 6, 8, 9, 12-14, 16 -18, and 20 are amended.
Claims 1-21 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 9, 12-15, 17-19 is/are rejected under 35 U.S.C. 103 as being anticipated by Magill et al. (CA 2397520) in view of Oz et al. (WO 2021229415).
Regarding claim 1:
Magill teaches:
A method comprising (Magill [Pg 8, Line 21-23]: In another aspect, the invention provides a method of performing a computer graphic character live.):
establishing, by a first computing device with a second computing device, a first communication session (Magill [Pg 9, Line 19]: a second computer coupled to the first computer);
receiving, by the first computing device, a first motion capture data stream originating from the second computing device during the communication session, the first motion capture data stream quantifying real-time movements of a first user of the second computing device (Magill [Pg 32, Line 25 – Pg 33, Line 5]: Performer movement information is received at the first computer from a manual input device that receives live manual manipulations and converts the manipulations into the performer movement information. Character motion information is created and stored based on combining the performer movement information with the character representation information. The character motion information is communicated in real time to a second computer.); and
rendering, by the first computing device to a display device, imagery of an animation of a three-dimensional (3D) (Magill [Pg 23, Line 28-30]: Thus, block 310 involves displaying a 3D or 2D animated CG character that moves in proportion to and in synchronicity with the movements of the input devices that occur in block 304.) model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user (Magill [Pg 26, Line 17-18]: Viewer 108 is a computer workstation, personal computer, or similar data processing system that serves as a rendering and display station for computer graphic characters that reproduce character positions performed by performer 102 using position transducers 104.).
Magill fails to teach:
comprising a conference between a first user of the second computing device and a second user of the first computing device;
wherein the 3D model of the first user is generated based on images of the first user.
Oz teaches:
comprising a conference between a first user of the second computing device and a second user of the first computing device (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment.);
wherein the 3D model of the first user is generated based on images of the first user (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0053] The generation of the first avatar and/or the inclusion of the first avatar may be responsive to information gained by the device of the first user or to a camera or sensor associated with the device of the first user. A non-limiting example of information may include information regarding the first participant and/or information regarding to the acquisition of images of the first participant (for example camera setting, illumination and/or ambient conditions).).
establishing, by the first computing device with a third computing device, a second communication session between the second user of the first computing device and a third user of the third computing device;merging, by the first computing device, the first communication session and the second communication session to generate a third communication session between the first user, the second user, and the third user (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
and a second motion capture data stream originating from the third computing device during the third communication session (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.),
the second motion capture data stream quantifying real-time movements of the third user of the third computing device;sending, by the first computing device to the third computing device, the first motion capture data stream;sending, by the first computing device to the second computing device, the second motion capture data stream (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 2:
Magill and Oz teach:
The method of claim 1 further comprising (as shown above):
receiving, by the first computing device from the second computing device during the communication session (as shown above), an audio stream comprising words spoken by the first user (Magill [Pg 74, Line 29]: A performance may be synchronized to a pre-recorded sound track.); and
presenting, on an audio device, the audio stream while concurrently rendering, to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user (Magill [Pg 74, Line 29 – Pg 75, Line 8]: A performance may be synchronized to a pre-recorded sound track. In one embodiment, character motion information can be captured, manipulated and played back, thereby creating recorded performance information, in coordination with character representation information, to effect computer generated performance of part, or all of a character in sync with a sound recording. Any other part of the character can be performed live while the recorded performance information runs or is played back. [Pg 16 Line 15-17] The second performance may be the final performance, further enhancing previously recorded, non-spontaneous performance such as lip-sync. The second performance may alternatively involve an additional character.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 3:
Magill and Oz teach:
The method of claim 1
wherein the first computing device does not receive a video stream from the second computing device during the communication session (Magill [Pg 28, Line 27]: Separate computers are not required.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 4:
Magill and Oz teach:
The method of claim 1 (as shown above) further comprising:
generating, by the first computing device during the communication session, a second motion capture data stream that quantifies real-time movements of a second user of the first computing device (Magill [Pg 34, Line 7 - 13]: Block 322 may involve using PCS 103B to create information that defines movement of a character during a second performance by performer 102 of transducers 104. The second performance may be the final performance, further enhancing previously recorded, non-spontaneous performance such as lip-sync.); and
sending, by the first computing device, the second motion capture data stream toward the second computing device (Magill [Pg 34, Line 16-20]: Computer 103 can concurrently send multiple Position values for multiple characters to viewer 108, which concurrently renders the multiple characters.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 5:
Magill and Oz teach:
The method of claim 1 further comprising:
determining, by the first computing device, a first user identifier that identifies the first user (Magill [Pg 19, Line 15]: Each expression may be associated with a symbolic label);
based on the first user identifier, selecting, by the first computing device, the 3D model of the first user from a plurality of 3D models (Magill [Pg 19, Line 19-21]: Thereafter, the performer can call up and re-use a character document containing complex expressions by selecting its label.) ; and
loading, by the first computing device, the 3D model into a memory of the first computing device (Magill [Pg 19, Line 17-19]: Information defining expressions is stored in non-volatile storage of computer 103.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 9:
Magill and Oz teach:
The method of claim 1 wherein establishing, by the first computing device with the second computing device (as shown above),
the communication session comprises directly contacting, by the first computing device, the second computing device (Magill [Pg 9, Line 19]: a second computer coupled to the first computer).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 12:
Magill teaches:
A first computing device (Magill [Pg 75, Line 18]: Computer system 700), comprising:
a memory (Magill [Pg 75, Line 18 - 24]: Computer system 700 also includes a main memory 706, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 702 for storing information and instructions to be executed by processor 704.); and
a processor device coupled to the memory operable to (Magill [Pg 75, Line 17-18]: …and a processor 704 coupled with bus 702 for processing information.):
establish, with a second computing device, a communication session (Magill [Pg 9, Line 19]: a second computer coupled to the first computer);
receive a first motion capture data stream originating from the second computing device during the communication session, the first motion capture data stream quantifying real-time movements of a first user of the second computing device (Magill [Pg 32, Line 25 – Pg 33, Line 5]: Performer movement information is received at the first computer from a manual input device that receives live manual manipulations and converts the manipulations into the performer movement information. Character motion information is created and stored based on combining the performer movement information with the character representation information. The character motion information is communicated in real time to a second computer.);
and render, to a display device, imagery of an animation of a three-dimensional (3D) model (Magill [Pg 33, Line 8-12]: Thus, block 310 involves displaying a 3D or 2D animated CG character that moves in proportion to and in synchronicity with the movements of the input devices that occur in block 304.) of the first user based on the first motion capture data stream that depicts the real-time movements of the first user (Magill [Pg 26, Line 17-18]: Viewer 108 is a computer workstation, personal computer, or similar data processing system that serves as a rendering and display station for computer graphic characters that reproduce character positions performed by performer 102 using position transducers 104.).
Magill fails to teach:
comprising a conference between a first user of the second computing device and a second user of the first computing device;
wherein the 3D model of the first user is generated based on images of the first user.
Oz teaches:
comprising a conference between a first user of the second computing device and a second user of the first computing device (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment.);
wherein the 3D model of the first user is generated based on images of the first user (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0053] The generation of the first avatar and/or the inclusion of the first avatar may be responsive to information gained by the device of the first user or to a camera or sensor associated with the device of the first user. A non-limiting example of information may include information regarding the first participant and/or information regarding to the acquisition of images of the first participant (for example camera setting, illumination and/or ambient conditions).).
establishing, by the first computing device with a third computing device, a second communication session between the second user of the first computing device and a third user of the third computing device;merging, by the first computing device, the first communication session and the second communication session to generate a third communication session between the first user, the second user, and the third user (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
and a second motion capture data stream originating from the third computing device during the third communication session (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.),
the second motion capture data stream quantifying real-time movements of the third user of the third computing device;sending, by the first computing device to the third computing device, the first motion capture data stream;sending, by the first computing device to the second computing device, the second motion capture data stream (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 13:
Magill and Oz teach:
The first computing device of claim 12 wherein the processor device is further operable to (as shown above):
receive, from the second computing device during the communication session, an audio stream (Magill [Pg 74, Line 29]: A performance may be synchronized to a pre-recorded sound track.); and
present, on an audio device, the audio stream while concurrently rendering, to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user (Magill [Pg 74, Line 29 – Pg 75, Line 8]: A performance may be synchronized to a pre-recorded sound track. In one embodiment, character motion information can be captured, manipulated and played back, thereby creating recorded performance information, in coordination with character representation information, to effect computer generated performance of part, or all of a character in sync with a sound recording. Any other part of the character can be performed live while the recorded performance information runs or is played back. [Pg 16 Line 15-17] The second performance may be the final performance, further enhancing previously recorded, non-spontaneous performance such as lip-sync. The second performance may alternatively involve an additional character.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 14:
Magill and Oz teach:
The first computing device of claim 12 wherein the processor device is further operable to (as shown above):
generate, during the communication session, a second motion capture data stream that quantifies real-time movements of a second user of the first computing device (Magill [Pg 34, Line 7 - 13]: Block 322 may involve using PCS 103B to create information that defines movement of a character during a second performance by performer 102 of transducers 104. The second performance may be the final performance, further enhancing previously recorded, non-spontaneous performance such as lip-sync.); and
send the second motion capture data stream toward the second computing device (Magill [Pg 34, Line 16-20]: Computer 103 can concurrently send multiple Position values for multiple characters to viewer 108, which concurrently renders the multiple characters.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 15:
Magill and Oz teach:
The first computing device of claim 12 wherein the processor device is further operable to (as shown above):
determine a first user identifier that identifies the first user (Magill [Pg 19, Line 15]: Each expression may be associated with a symbolic label);
based on the first user identifier, select the 3D model of the first user from a plurality of 3D models (Magill [Pg 19, Line 19-21]: Thereafter, the performer can call up and re-use a character document containing complex expressions by selecting its label.); and
load the 3D model into the memory of the first computing device (Magill [Pg 19, Line 17-19]: Information defining expressions is stored in non-volatile storage of computer 103.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 17:
Magill and Oz teach:
A non-transitory computer-readable storage medium that includes executable instructions operable to cause a processor device of a first computing device to (Magill [Pg 75, Line 18 - 24]: Computer system 700 also includes a main memory 706, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 702 for storing information and instructions to be executed by processor 704.):
establish, with a second computing device, a communication session (Magill [Pg 9, Line 19]: a second computer coupled to the first computer);
receive a first motion capture data stream originating from the second computing device during the communication session, the first motion capture data stream quantifying real-time movements of a first user of the second computing device (Magill [Pg 32, Line 25 – Pg 33, Line 5]: Performer movement information is received at the first computer from a manual input device that receives live manual manipulations and converts the manipulations into the performer movement information. Character motion information is created and stored based on combining the performer movement information with the character representation information. The character motion information is communicated in real time to a second computer.); and
render, to a display device, imagery of an animation of a three-dimensional (3D) model (Magill [Pg 33, Line 8-12]: Thus, block 310 involves displaying a 3D or 2D animated CG character that moves in proportion to and in synchronicity with the movements of the input devices that occur in block 304.) of the first user based on the first motion capture data stream that depicts the real-time movements of the first user (Magill [Pg 26, Line 17-18]: Viewer 108 is a computer workstation, personal computer, or similar data processing system that serves as a rendering and display station for computer graphic characters that reproduce character positions performed by performer 102 using position transducers 104.).
Magill fails to teach:
comprising a conference between a first user of the second computing device and a second user of the first computing device;
wherein the 3D model of the first user is generated based on images of the first user.
Oz teaches:
comprising a conference between a first user of the second computing device and a second user of the first computing device (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment.);
wherein the 3D model of the first user is generated based on images of the first user (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0053] The generation of the first avatar and/or the inclusion of the first avatar may be responsive to information gained by the device of the first user or to a camera or sensor associated with the device of the first user. A non-limiting example of information may include information regarding the first participant and/or information regarding to the acquisition of images of the first participant (for example camera setting, illumination and/or ambient conditions).).
establishing, by the first computing device with a third computing device, a second communication session between the second user of the first computing device and a third user of the third computing device;merging, by the first computing device, the first communication session and the second communication session to generate a third communication session between the first user, the second user, and the third user (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
and a second motion capture data stream originating from the third computing device during the third communication session (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.),
the second motion capture data stream quantifying real-time movements of the third user of the third computing device;sending, by the first computing device to the third computing device, the first motion capture data stream;sending, by the first computing device to the second computing device, the second motion capture data stream (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 18:
Magill and Oz teach:
The non-transitory computer-readable storage medium of claim 17 wherein the instructions are further operable to cause the processor device to (as shown above):
generate, during the communication session, a second motion capture data stream that quantifies real-time movements of a second user of the first computing device (Magill [Pg 34, Line 7 - 13]: Block 322 may involve using PCS 103B to create information that defines movement of a character during a second performance by performer 102 of transducers 104. The second performance may be the final performance, further enhancing previously recorded, non-spontaneous performance such as lip-sync.); and
send the second motion capture data stream toward the second computing device (Magill [Pg 34, Line 16-20]: Computer 103 can concurrently send multiple Position values for multiple characters to viewer 108, which concurrently renders the multiple characters.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Regarding claim 19:
Magill and Oz teach:
The non-transitory computer-readable storage medium of claim 17 wherein the instructions are further operable to cause the processor device to (as shown above):
determine a first user identifier that identifies the first user (Magill [Pg 19, Line 15]: Each expression may be associated with a symbolic label);
based on the first user identifier, select the 3D model of the first user from a plurality of 3D models (Magill [Pg 19, Line 19-21]: Thereafter, the performer can call up and re-use a character document containing complex expressions by selecting its label.); and
load the 3D model into a memory of the first computing device (Magill [Pg 19, Line 17-19]: Information defining expressions is stored in non-volatile storage of computer 103.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Oz. Having a call between two users that is a model of a user that is generated by taking pictures of the user, as in Oz, would benefit the Magill teachings by allowing for users to call each other using animated models. Additionally, this is the application of a known technique, having a call between two users that is a model of a user that is generated by taking pictures of the user, to yield predictable results.
Claim(s) 6, 16, 20 is/are rejected under 35 U.S.C. 103 as being un patentable over Magill et al. (CA 2397520) in view of Oz et al. (WO 2021229415) and Shen et al. (WO 2019095131).
Regarding claim 6:
Magill and Oz teach:
The method of claim 1 further comprising (as shown above):
wherein rendering, by the first computing device to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user further comprises (as shown above):
Magill and Oz does not teach:
receiving, by the first computing device, a second motion capture data stream originating from a third computing device during the communication session, the second motion capture data stream quantifying real-time movements of a second user of the third computing device; and
concurrently rendering, by the first computing device to the display device, imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user and imagery of an animation of a 3D model of the second user based on the second motion capture data stream that depicts the real-time movements of the second user.
Shen teaches:
receiving, by the first computing device, a second motion capture data stream originating from a third computing device during the communication session, the second motion capture data stream quantifying real-time movements of a second user of the third computing device (Shen [Pg 2]: the action of the corresponding second avatar in the virtual space, and displaying the second avatar in the virtual space by the head display device connected by the first client; wherein the second user The wearer of the motion capture device connected to the second client, the third action data is action data of the second avatar.); and
concurrently rendering, by the first computing device to the display device, imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user and imagery of an animation of a 3D model of the second user based on the second motion capture data stream that depicts the real-time movements of the second user (Shen [Pg 6]: For example, if there are three users, the second client has two, and the first client displays the second avatar corresponding to the second user corresponding to the two second clients through the connected display device. Of course, each of the second clients also displays the first avatar and the second avatar corresponding to the second user corresponding to the other second client through the connected head display device.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill and Oz with Shen, having multiple motion captures for multiple different users that is then displayed, as in Shen, would benefit the Magill and Oz teachings by allowing the use of multiple different users in the system. Additionally, this is the application of a known technique, taking and rendering different motion capture data, to yield predictable results.
Regarding claim 16:
Magill and Oz teach:
The first computing device of claim 12 wherein the processor device is further operable to:
and wherein to render, to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user, the processor device is further to:
Magill does not teach:
receive a second motion capture data stream originating from a third computing device during the communication session, the second motion capture data stream quantifying real-time movements of a second user of the third computing device;
concurrently render, to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user and imagery of an animation of a 3D model of the second user based on the second motion capture data stream that depicts the real-time movements of the second user.
Shen teaches:
receive a second motion capture data stream originating from a third computing device during the communication session, the second motion capture data stream quantifying real-time movements of a second user of the third computing device (Shen [Pg 2]: the action of the corresponding second avatar in the virtual space, and displaying the second avatar in the virtual space by the head display device connected by the first client; wherein the second user The wearer of the motion capture device connected to the second client, the third action data is action data of the second avatar.);
concurrently render, to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user and imagery of an animation of a 3D model of the second user based on the second motion capture data stream that depicts the real-time movements of the second user (Shen [Pg 6]: For example, if there are three users, the second client has two, and the first client displays the second avatar corresponding to the second user corresponding to the two second clients through the connected display device. Of course, each of the second clients also displays the first avatar and the second avatar corresponding to the second user corresponding to the other second client through the connected head display device.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill and Oz with Shen, having multiple motion captures for multiple different users that is then displayed, as in Shen, would benefit the Magill and Oz teachings by allowing the use of multiple different users in the system. Additionally, this is the application of a known technique, taking and rendering different motion capture data, to yield predictable results.
Regarding claim 20:
Magill and Oz teach
The non-transitory computer-readable storage medium of claim 17 wherein the instructions are further operable to cause the processor device to:
and wherein to render, by the first computing device to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user, the processor device is further to:
Magill and Oz does not teach:
receive a second motion capture data stream originating from a third computing device during the communication session, the second motion capture data stream quantifying real-time movements of a second user of the third computing device;
concurrently render, by the first computing device to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user and imagery of an animation of a 3D model of the second user based on the second motion capture data stream that depicts the real-time movements of the second user.
Shen teaches:
receive a second motion capture data stream originating from a third computing device during the communication session, the second motion capture data stream quantifying real-time movements of a second user of the third computing device (Shen [Pg 2]: the action of the corresponding second avatar in the virtual space, and displaying the second avatar in the virtual space by the head display device connected by the first client; wherein the second user The wearer of the motion capture device connected to the second client, the third action data is action data of the second avatar.);
concurrently render, by the first computing device to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user and imagery of an animation of a 3D model of the second user based on the second motion capture data stream that depicts the real-time movements of the second user (Shen [Pg 6]: For example, if there are three users, the second client has two, and the first client displays the second avatar corresponding to the second user corresponding to the two second clients through the connected display device. Of course, each of the second clients also displays the first avatar and the second avatar corresponding to the second user corresponding to the other second client through the connected head display device.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill and Oz with Shen, having multiple motion captures for multiple different users that is then displayed, as in Shen, would benefit the Magill and Oz teachings by allowing the use of multiple different users in the system. Additionally, this is the application of a known technique, taking and rendering different motion capture data, to yield predictable results.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being un patentable over
Magill et al. (CA 2397520) in view of Oz et al. (WO 2021229415) and Kim et al. (WO 2021251534 A1).
Regarding claim 7:
Magill and Oz teach:
The method of claim 1 wherein the first motion capture data stream (as shown above)
Magill does not teach:
identifies one or more of facial movements of the first user and hand movements of the first user
Kim teaches:
identifies one or more of facial movements of the first user and hand movements of the first user (Kim [Pg 8]: In the case of 11 neurons as shown in (a), they are attached only to the arms, legs, and torso, excluding the hands and feet, so that motion capture data can be tracked. [Pg 1]: The method for providing the real-time broadcasting platform, of the present invention, comprises the steps of: extracting, in real time, a user's motion capture data and face capture data on the basis of a motion detection sensor mounted on the user's body and an image captured by the user, respectively; generating animations of a preset character which represent a motion corresponding the extracted motion capture data and represent a facial expression corresponding to rigging data to which the face capture data is applied; and providing a broadcasting platform for transmitting the generated animations in real time.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill and Oz with Kim. Using face and hand data, as in Kim, would benefit the Magill and Oz teachings by Allowing for more accurate tracking and motion data. Additionally, this is the application of a known technique, using multiple different parts of a person like face and hand data, to yield predictable results.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being un patentable over
Magill et al. (CA 2397520) in view of Oz et al. (WO 2021229415) and Jong et al. (US 20110115878).
Regarding claim 8:
Magill and Oz teach:
The method of claim 1 wherein establishing, by the first computing device with the second computing device (as shown above),
Magill and Oz does not teach:
the communication session comprises connecting, by the first computing device, to a mixer that is operable to receive first audio communications from the first computing device and provide the first audio communications to the second computing device, and to receive second audio communications from the second computing device and provide second audio the communications to the first computing device
Jong teaches:
the communication session comprises connecting, by the first computing device, to a mixer that is operable to receive first audio communications from the first computing device and provide the first audio communications to the second computing device, and to receive second audio communications from the second computing device and provide second audio the communications to the first computing device (Jong [0005]: A system is disclosed enabling remote participation in a live proceeding that includes a first computer configured to transmit and receive audio and video data with a second computer through a network. An audio mixer is configured to combine a first audio signal received from a microphone with a second audio signal received from the first computer to provide a third audio signal. A capture device is coupled to the audio mixer and to the first computer.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill with Jong. Having a system that allows for the transmission of sound data, as in Jong, would benefit the Magill teachings by allowing the motion data to have sound that can play. Additionally, this is the application of a known technique, having a system that can transmit audio, to yield predictable results.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being un patentable over Magill et al. (CA 2397520) in view of Oz et al. (WO 2021229415) and Corazza et al. (US 20100285877).
Regarding claim 10:
Magill and Oz teach:
The method of claim 1 wherein the display device comprises
Magill does not teach:
a holographic display device and wherein rendering, by the first computing device to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user comprises:
rendering, by the first computing device to the holographic display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user.
Corazza teaches:
a holographic display device and wherein rendering, by the first computing device to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user comprises (Corazza [0023]: FIG. 1 is a block diagram illustrating a system for performing remote markerless motion capture to drive 3D animation in real time in accordance with an embodiment of the invention.):
rendering, by the first computing device to the holographic display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user (Corazza [0043]: In many embodiments, the rendered 3D character animations are displayed to the performer through a 3D/virtual reality device that can be worn on the performer's body (e.g. virtual reality goggles) or a standalone device (e.g. a 3D television or holographic display).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill and Oz with Corazza. Displaying the 3d animations on a holographic display, as in Corazza, would benefit the Magill and Oz teachings by allowing for a different way to display the animation to provide a more realistic view. Additionally, this is the application of a known technique, displaying 3d animation on a holographic display, to yield predictable results.
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being un patentable over Magill et al. (CA 2397520) in view of Yang et al. (CN 111479103 A).
Regarding claim 11:
Magill and Oz teach:
The method of claim 1 wherein the display device comprises
by the first computing device to the display device, the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user comprises (as shown above):
the imagery of the animation of the 3D model of the first user based on the first motion capture data stream that depicts the real-time movements of the first user (as shown above).
Magill and Oz does not teach:
a light field display device and wherein rendering,
rendering, by the first computing device to the light field display device,
Yang teaches:
a light field display device and wherein rendering (Yang [Pg 2]: A near-eye light field display system, comprising an image emitter, an eye tracker, a controller and a diffraction optical device),
rendering, by the first computing device to the light field display device (Yang [Pg 2]: A near-eye light field display system, comprising an image emitter, an eye tracker, a controller and a diffraction optical device, wherein: the image emitter is connected with the controller, for generating the viewpoint image composed of image light, and emitting at least two viewpoint images to the diffraction optical device according to the angle indicated by the control signal of the controller; the eye tracker is connected with the controller, for real-time monitoring the movement of the human eye, to obtain the real-time human eye position information, and the human eye position information is sent to the controller),
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill and Oz with Yang. Displaying onto a light field display, as in Yang, would benefit the Magill and Oz teachings by would benefit the Magill teachings by allowing for a different way to display the animation to provide a more realistic view. Additionally, this is the application of a known technique, displaying 3d animation on a light field display, to yield predictable results.
Claim(s) 21 is/are rejected under 35 U.S.C. 103 as being un patentable over Magill et al. (CA 2397520) in view of Johnson et al. (US 20120154582).
Regarding claim 21:
Magill and Oz teach:
The method of claim 1,
Magill and Oz fails to teach:
wherein the first motion capture data stream comprises data generated at least in part based on a depth sensor operable to measure a human feature in 3D space (Johnson [0152] Embodiments of the present invention consider the motion features extracted in the space-time volumes of captured video imageries and potential other assisted sensors. The space-time motion features that can be used to characterize the training samples include, as examples, motion features from spatial-temporal filtering, motion features from sparse interest operators, occupancy measures from 3D image and depth sensors, positioning measures from bed load sensors (specifically for the recognition of patient actions in the bed), body articulation features from articulated human model fitting (location of head, arms, torso, legs, etc.), motion features from object detectors, positions of medical equipments (ventilator and medical gloves for ventilator-associated pneumonia prevention), and facial features from face model fitting and alignment (location of eyes, eyebrows, mouth, etc., for the measures of patient pain, distress and/or stroke).).
Johnson teaches:
wherein the first motion capture data stream comprises data generated at least in part based on a depth sensor operable to measure a human feature in 3D space (Johnson [0152] Embodiments of the present invention consider the motion features extracted in the space-time volumes of captured video imageries and potential other assisted sensors. The space-time motion features that can be used to characterize the training samples include, as examples, motion features from spatial-temporal filtering, motion features from sparse interest operators, occupancy measures from 3D image and depth sensors, positioning measures from bed load sensors (specifically for the recognition of patient actions in the bed), body articulation features from articulated human model fitting (location of head, arms, torso, legs, etc.), motion features from object detectors, positions of medical equipments (ventilator and medical gloves for ventilator-associated pneumonia prevention), and facial features from face model fitting and alignment (location of eyes, eyebrows, mouth, etc., for the measures of patient pain, distress and/or stroke).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Magill and Oz with Johnson. Having a depth sensor to capture human features, as in Johnson, would benefit the Magill and Oz teachings by allowing for users adjust models based on depth. Additionally, this is the application of a known technique, having a depth sensor to capture human features, to yield predictable results.
Response to Arguments
Applicant's arguments filed 03/09/2026 have been fully considered but they are not persuasive.
Applicant has amended the independent claims, however the amendments are taught by Oz.
Applicant has included the following amendments:
establishing, by the first computing device with a third computing device, a second communication session between the second user of the first computing device and a third user of the third computing device;merging, by the first computing device, the first communication session and the second communication session to generate a third communication session between the first user, the second user, and the third user (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
and a second motion capture data stream originating from the third computing device during the third communication session (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.),
the second motion capture data stream quantifying real-time movements of the third user of the third computing device;sending, by the first computing device to the third computing device, the first motion capture data stream;sending, by the first computing device to the second computing device, the second motion capture data stream (Oz [0049] For example, referring to a 3D video conference that involves multiple participants. A first participant is imaged, and a second participant wishes to view a first avatar (or any other 3D visual representation) of the first participant within a virtual 3D video conference environment. [0050] The generation of the first avatar (or any other 3D visual representation) may be executed in various manners - for example - only by a device of the second participant, only by the device of the first participant, partially by the device of the second participant, partially by the device of the first participant, by a cooperation between the devices of the first and second participants, by another computerized system (such as but not limited to a cloud system or a remote system), and/or any combination of one or more devices. [0069] The one or more additional steps may include at least one out of: a. Determining a field of view of a third participant within the virtual 3D video conference environment. b. Setting a third updated representation of the virtual 3D video conference environment that may be sent to a third participant device to reflect the field of view of the third participant.);
Oz has been updated to reflect these updates. Oz includes multiple participants which just simply means more than one. It also includes language about a third participant. Figure 11 shows the multiple participants that can be included. It would be obvious to one well versed in the art that a group call could be a merged call. This would in turn allow for more than 2 users to be in a call and see each other’s avatars therefore allowing for the data from one’s camera to be merged or transmitted to another.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENIS VASILIY MINKO whose telephone number is (571)270-5226. The examiner can normally be reached Monday-Thursday 8:30-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENIS VASILIY MINKO/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights