DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 2-10, 14-17, 19, and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if the claims are rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
In regards to dependent claim 2, none of the cited prior art alone or in combination provides motivation to teach “wherein the image capture device includes a real camera, and wherein the obtaining interaction data of at least two real characters performing interaction with each other in a target scene comprises: obtaining, by the image rendering device, original data that is recorded by the real camera of the at least two real characters performing interaction with each other in the target scene, wherein the real camera is mounted on a mechanical arm, and the mechanical arm is connected to the real camera through an end actuator of the mechanical arm; determining, by the image rendering device, a first coordinate conversion relationship between the real camera and the mechanical arm, and a second coordinate conversion relationship between the mechanical arm and a virtual camera, wherein the image rendering device includes the virtual camera and the virtual camera is configured for performing the picture rendering; and mapping, by the image rendering device, the original data based on the first coordinate conversion relationship and the second coordinate conversion relationship, to obtain mapped data of the original data in the virtual camera, and determining the mapped data as the interaction data” as the references only teach techniques for collecting motion capture data for tracking users in a real environment and rendering virtual characters in addition to coordinate transformation feature for robotic arms and their connected cameras, however the references fail to explicitly disclose determination of first and second coordinate conversions between the real camera and a mechanical arm connected to the camera and subsequently the mechanical arm and a virtual camera for mapping data to determine the interaction data, in conjunction with the features of claim 1 with which it depends for the purpose of rendering virtual characters from real characters in the same scene based on motion capture and interaction data.
In addition, there is no teaching, suggestion, or motivation found in the current references and none that can be inferred from the examiner’s own knowledge with respect to the current limitation.
In regards to dependent claims 14 and 18, these claims recite limitations similar in scope to claim 2, and thus are objected to based on the same rationale as provided above.
In regards to dependent claims 3-10, 15-17, 19, and 20, these claims depend from objected to base claims, and thus are objected to based on the same rationale as provided above.
As allowable subject matter has been indicated, applicant's reply must either comply with all formal requirements or specifically traverse each requirement not complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 11, 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Peiying (CN 107274466 A, hereinafter referenced “Pei”) in view of Bradski (US 2016/0026253 A1, hereinafter referenced “Bradski”).
In regards to claim 1. Pei discloses a method for electronically rendering an interaction picture comprising a virtual character (Pei, para [0007]), the method comprising:
-obtaining, by an image rendering device from an image capture device, interaction data of at least two real characters performing interaction with each other in a target scene and motion-capture data of a first character in the at least two real characters (Pei, para [0034] and [0051] and [0052]; Reference at [0034] discloses Refer to the flowchart of the real-time full-body motion capture method shown in Figure 1. the method includes the following main steps: A. Creating a virtual 3D character and importing it into a graphics engine to generate a character model; B. Capturing real actor motion information in real time through video and/or wearable devices (i.e. obtaining, by an image rendering device from an image capture device, interaction data of at least two real characters performing interaction with each other in a target scene), and then associating and controlling the character model. Para [0051] discloses S501. Select an actor and adjust the motion capture equipment for them to capture their body movements, facial expressions, and voice (i.e. getting motion of actors interpreted as motion-capture data of a first character in the at least two real characters). Actors can conduct action and facial expression tests based on pre-planned lines and scripts),
-the motion-capture data being configured for determining a movement of the virtual character (Pei, para [0034]; Reference discloses B. Capturing real actor motion information in real time through video and/or wearable devices, and then associating and controlling the character model; C. Synthesizing the character model into an animation scene (i.e. real actor motion used for controlling character model interpreted as motion-capture data being configured for determining a movement of the virtual character)),
-the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character (Pei, para [0048]; Reference discloses Furthermore, step B includes: capturing the actor's body movements and/or facial expressions, converting them into body movement data, facial movement data, and character mixing data associated with the character design characteristics of the 3D character, then associating them with the corresponding character model in the graphics engine, and configuring the actor's body movements and/or facial expressions to be synchronized in real time with the body movements and/or facial expressions of the character model animation (i.e. synchronizing of actors movements with the character interpreted as the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character );
Pei does not explicitly disclose but Bradski teaches
and performing, by the image rendering device, picture rendering based on the interaction data and the motion-capture data, to generate an image in which the virtual character performs interaction with a second character in the target scene, the second character being a character other than the first character in the at least two real characters (Bradski, para [0868]-[0869]; Reference at [0868] discloses for example, a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode. Para [0869] discloses that FIG. 57A illustrates an example wherein a first user 5701 (interfacing a digital world of the AR system in a blended virtual interface mode) and first object 5702 appear as virtual objects to a second user 5722 interfacing the same digital world of the AR system in a full virtual reality mode….The first user 5701 may be scanned, for example, by a motion capture system or similar device (i.e. and performing, by the image rendering device, picture rendering based on the interaction data and the motion-capture data), and be rendered in the virtual world as a first rendered physical object 5731 (i.e. generate an image in which the virtual character performs interaction with a second character in the target scene, the second character being a character other than the first character in the at least two real characters).
Pei and Bradski are combinable because they are in the same field of endeavor regarding virtual object rendering. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the real-time body motion capture of Pei to include the virtual and augmented reality creation features of Bradski in order to provide the user with a system for real-time full-body motion capture for creating a virtual 3D character, capturing real word actor motion data, and synthesizing the character in the animation scene aligning its movements with the real world actor as taught by Pei while incorporating the virtual and augmented reality creation features of Bradski to allow for use of various configurations for presenting virtual reality and augmented reality experiences to facilitate interactions amongst users via motion capture data where a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode, thus allowing for greater system flexibility, applicable to the virtual rendering systems as taught in Pei.
In regards to claim 11. Pei in view Bradski teach the method according to claim 1.
Pei further discloses
-wherein the obtaining motion-capture data of a first character in the at least two real characters comprises: obtaining, when the first character in the at least two real characters wears a motion capture device, position change data that is of a marked point on the motion capture device and that is captured by a real camera, and using the position change data as motion data of the first character, wherein the marked point corresponds to a skeletal key point of the first character (Pei, para [0034], [0042], and [0043]; Reference discloses B. Capturing real actor motion information in real time through video and/or wearable devices, and then associating and controlling the character model; C. Synthesizing the character model into an animation scene. Para [0042]-[0043] discloses S302, Bind and create weights for the animation skeletal model. The skeletal animation here uses motion capture technology to vividly depict the movement of the real human body (i.e. . All skeletal models are stored in the motion module database));
-obtaining, when the first character is mounted with an expression capture device, expression data that is of the first character and that is captured by the expression capture device (Pei, para [0034]; Reference discloses the method includes the following main steps: A. Creating a virtual 3D character and importing it into a graphics engine to generate a character model; B. Capturing real actor motion information in real time through video and/or wearable devices, and then associating and controlling the character model; C. Synthesizing the character model into an animation scene, configuring physical dynamic parameters for the character model according to the environmental parameters of the animation scene, and then controlling the animation scene through the control panel in the client backend; wherein, the motion information includes body movements and/or facial expressions));
-obtaining, when the first character is mounted with a voice collection device, voice data that is of the first character and that is collected by the voice collection device; and determining at least one of the motion data, the expression data, and the voice data that are of the first character as the motion-capture data (Pei, [0051]; Reference discloses S501. Select an actor and adjust the motion capture equipment for them to capture their body movements, facial expressions, and voice. Actors can conduct action and facial expression tests based on pre-planned lines and scripts. Here, the actor's voice needs to be processed and converted into a tone that matches the 3D character design).
In regards to claim 13. Pei discloses an apparatus for rendering an interaction picture (Pei, para [0007]), the apparatus comprising:
-a processor (Pei, para [0055]; Reference discloses microprocessor or other data processor to implement the steps described above);
-and memory storing computer readable instructions that, when executed by the processor, cause the apparatus to (Pei, para [0055]-[0056]; Reference discloses microprocessor or other data processor to implement the steps described above and computer programs can be applied in non-volatile memory ):
-obtain, from an image capture device, interaction data of at least two real characters performing interaction with each other in a target scene and motion-capture data of a first character in the at least two real characters (Pei, para [0034] and [0051] and [0052]; Reference at [0034] discloses Refer to the flowchart of the real-time full-body motion capture method shown in Figure 1. the method includes the following main steps: A. Creating a virtual 3D character and importing it into a graphics engine to generate a character model; B. Capturing real actor motion information in real time through video and/or wearable devices (i.e. obtaining, by an image rendering device from an image capture device, interaction data of at least two real characters performing interaction with each other in a target scene), and then associating and controlling the character model. Para [0051] discloses S501. Select an actor and adjust the motion capture equipment for them to capture their body movements, facial expressions, and voice (i.e. getting motion of actors interpreted as motion-capture data of a first character in the at least two real characters). Actors can conduct action and facial expression tests based on pre-planned lines and scripts),
-the motion-capture data being configured for determining a movement of a virtual character (Pei, para [0034]; Reference discloses B. Capturing real actor motion information in real time through video and/or wearable devices, and then associating and controlling the character model; C. Synthesizing the character model into an animation scene (i.e. real actor motion used for controlling character model interpreted as motion-capture data being configured for determining a movement of the virtual character)),
-the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character (Pei, para [0048]; Reference discloses Furthermore, step B includes: capturing the actor's body movements and/or facial expressions, converting them into body movement data, facial movement data, and character mixing data associated with the character design characteristics of the 3D character, then associating them with the corresponding character model in the graphics engine, and configuring the actor's body movements and/or facial expressions to be synchronized in real time with the body movements and/or facial expressions of the character model animation (i.e. synchronizing of actors movements with the character interpreted as the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character);
Pei does not explicitly disclose but Bradski teaches
-and perform picture rendering based on the interaction data and the motion-capture data, to generate an image in which the virtual character performs interaction with a second character in the target scene, the second character being a character other than the first character in the at least two real characters (Bradski, para [0868]-[0869]; Reference at [0868] discloses for example, a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode. Para [0869] discloses that FIG. 57A illustrates an example wherein a first user 5701 (interfacing a digital world of the AR system in a blended virtual interface mode) and first object 5702 appear as virtual objects to a second user 5722 interfacing the same digital world of the AR system in a full virtual reality mode….The first user 5701 may be scanned, for example, by a motion capture system or similar device (i.e. and performing, by the image rendering device, picture rendering based on the interaction data and the motion-capture data), and be rendered in the virtual world as a first rendered physical object 5731 (i.e. generate an image in which the virtual character performs interaction with a second character in the target scene, the second character being a character other than the first character in the at least two real characters).
Pei and Bradski are combinable because they are in the same field of endeavor regarding virtual object rendering. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the real-time body motion capture of Pei to include the virtual and augmented reality creation features of Bradski in order to provide the user with a system for real-time full-body motion capture for creating a virtual 3D character, capturing real word actor motion data, and synthesizing the character in the animation scene aligning its movements with the real world actor as taught by Pei while incorporating the virtual and augmented reality creation features of Bradski to allow for use of various configurations for presenting virtual reality and augmented reality experiences to facilitate interactions amongst users via motion capture data where a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode, thus allowing for greater system flexibility, applicable to the virtual rendering systems as taught in Pei.
In regards to claim 18. Pei discloses a non-transitory computer-readable medium storing computer-readable instructions that, when executed, cause an apparatus to (Pei, para [0056]):
-obtain, from an image capture device, interaction data of at least two real characters performing interaction with each other in a target scene and motion-capture data of a first character in the at least two real characters (Pei, para [0034] and [0051] and [0052]; Reference at [0034] discloses Refer to the flowchart of the real-time full-body motion capture method shown in Figure 1. the method includes the following main steps: A. Creating a virtual 3D character and importing it into a graphics engine to generate a character model; B. Capturing real actor motion information in real time through video and/or wearable devices (i.e. obtaining, by an image rendering device from an image capture device, interaction data of at least two real characters performing interaction with each other in a target scene), and then associating and controlling the character model. Para [0051] discloses S501. Select an actor and adjust the motion capture equipment for them to capture their body movements, facial expressions, and voice (i.e. getting motion of actors interpreted as motion-capture data of a first character in the at least two real characters). Actors can conduct action and facial expression tests based on pre-planned lines and scripts),
-the motion-capture data being configured for determining a movement of a virtual character (Pei, para [0034]; Reference discloses B. Capturing real actor motion information in real time through video and/or wearable devices, and then associating and controlling the character model; C. Synthesizing the character model into an animation scene (i.e. real actor motion used for controlling character model interpreted as motion-capture data being configured for determining a movement of the virtual character)),
-the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character (Pei, para [0048]; Reference discloses furthermore, step B includes: capturing the actor's body movements and/or facial expressions, converting them into body movement data, facial movement data, and character mixing data associated with the character design characteristics of the 3D character, then associating them with the corresponding character model in the graphics engine, and configuring the actor's body movements and/or facial expressions to be synchronized in real time with the body movements and/or facial expressions of the character model animation (i.e. synchronizing of actors movements with the character interpreted as the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character);
Pei does not explicitly disclose but Bradski teaches
-and perform picture rendering based on the interaction data and the motion-capture data, to generate an image in which the virtual character performs interaction with a second character in the target scene, the second character being a character other than the first character in the at least two real characters (Bradski, para [0868]-[0869]; Reference at [0868] discloses for example, a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode. Para [0869] discloses that FIG. 57A illustrates an example wherein a first user 5701 (interfacing a digital world of the AR system in a blended virtual interface mode) and first object 5702 appear as virtual objects to a second user 5722 interfacing the same digital world of the AR system in a full virtual reality mode….The first user 5701 may be scanned, for example, by a motion capture system or similar device (i.e. and performing, by the image rendering device, picture rendering based on the interaction data and the motion-capture data), and be rendered in the virtual world as a first rendered physical object 5731 (i.e. generate an image in which the virtual character performs interaction with a second character in the target scene, the second character being a character other than the first character in the at least two real characters).
Pei and Bradski are combinable because they are in the same field of endeavor regarding virtual object rendering. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the real-time body motion capture of Pei to include the virtual and augmented reality creation features of Bradski in order to provide the user with a system for real-time full-body motion capture for creating a virtual 3D character, capturing real word actor motion data, and synthesizing the character in the animation scene aligning its movements with the real world actor as taught by Pei while incorporating the virtual and augmented reality creation features of Bradski to allow for use of various configurations for presenting virtual reality and augmented reality experiences to facilitate interactions amongst users via motion capture data where a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode, thus allowing for greater system flexibility, applicable to the virtual rendering systems as taught in Pei.
Claims 12 is rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Peiying (CN 107274466 A) in view of Bradski (US 2016/0026253 A1) as applied to claim 1 above, and further in view of McTernan (US 2023/0326092 A1, hereinafter referenced “McTernan”).
In regards to claim 12. Pei in view Bradski teach the method according to claim 1.
Pei further discloses
-wherein the performing picture rendering based on the interaction data and the motion-capture data, to obtain a picture in which the virtual character performs interaction with a second character in the target scene comprises: performing the picture rendering based on the interaction data, to obtain a first interaction picture in which the at least two real characters perform interaction with each other in the target scene (Pei, para [0034] and [0051] and [0052]; Reference at [0034] discloses refer to the flowchart of the real-time full-body motion capture method shown in Figure 1. the method includes the following main steps: A. Creating a virtual 3D character and importing it into a graphics engine to generate a character model; B. Capturing real actor motion information in real time through video and/or wearable devices (i.e. obtaining, by an image rendering device from an image capture device, interaction data of at least two real characters performing interaction with each other in a target scene), and then associating and controlling the character model. Para [0051] discloses S501. Select an actor and adjust the motion capture equipment for them to capture their body movements, facial expressions, and voice);
Pei does not explicitly disclose but Bradski teaches
-
-and generating the virtual character corresponding to the first character Bradski, para [0868]-[0869]; Reference at [0868] discloses for example, a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode. Para [0869] discloses that FIG. 57A illustrates an example wherein a first user 5701 (interfacing a digital world of the AR system in a blended virtual interface mode) and first object 5702 appear as virtual objects to a second user 5722 interfacing the same digital world of the AR system in a full virtual reality mode….The first user 5701 may be scanned, for example, by a motion capture system or similar device (i.e. the interaction data and the motion-capture data), and be rendered in the virtual world as a first rendered physical object 5731 (i.e. generate an image in which the virtual character performs interaction with a second character in the target scene).
Pei and Bradski does not disclose but McTernan teaches
-creating a space mask corresponding to the first character, and masking the first character in the first interaction picture based on the space mask, to obtain a second interaction picture in which the second character performs interaction with the space mask (McTernan, para [0055]; Reference discloses in another implementation, the emoji may be superimposed over the face of the respective human user's avatar in the virtual environment. In yet other implementations, there may be an option to show or hide the emojis, and this option can be exercised by either the leader of the meeting, for example, or by one or more of the audience members or other participants individually. (The showing or hiding of emoji’s which can be placed over user’s avatar representations in interactive environment interpreted as analogous to having a space mask corresponding to the characters to obtain a second interaction picture in which the second character performs interaction with the space mask));
-(first character) at a position of the space mask (McTernan, para [0055]; Reference discloses in another implementation, the emoji may be superimposed over the face of the respective human user's avatar in the virtual environment. In yet other implementations, there may be an option to show or hide the emojis, and this option can be exercised by either the leader of the meeting, for example, or by one or more of the audience members or other participants individually. The showing or hiding of emoji’s which can be placed over user’s avatar representations in interactive environment interpreted as analogous to having a space mask corresponding to the characters to obtain a second interaction picture in which the second character performs interaction with the space mask))
Pei and Bradski are combinable because they are in the same field of endeavor regarding virtual object rendering. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the real-time body motion capture of Pei to include the virtual and augmented reality creation features of Bradski in order to provide the user with a system for real-time full-body motion capture for creating a virtual 3D character, capturing real word actor motion data, and synthesizing the character in the animation scene aligning its movements with the real world actor as taught by Pei while incorporating the virtual and augmented reality creation features of Bradski to allow for use of various configurations for presenting virtual reality and augmented reality experiences to facilitate interactions amongst users via motion capture data where a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode, thus allowing for greater system flexibility, applicable to the virtual rendering systems as taught in Pei.
Pei and McTernan are also combinable because they are in the same field of endeavor regarding virtual object rendering. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the real-time body motion capture of Pei, in view of the virtual and augmented reality creation features of Bradski, to include the real-time visualization features of McTernan in order to provide the user with a system for real-time full-body motion capture for creating a virtual 3D character, capturing real word actor motion data, and synthesizing the character in the animation scene aligning its movements with the real world actor as taught by Pei while incorporating the virtual and augmented reality creation features of Bradski to allow for use of various configurations for presenting virtual reality and augmented reality experiences to facilitate interactions amongst users via motion capture data where a first user interfacing a particular virtual world in a virtual interface mode may interact with a second user interfacing the same virtual world in an augmented reality mode. Further incorporating the real-time visualization features of McTernan allows for use of a HMD to obtain motion data which is then used to generate reactions in the form of emoji placed near or on top of a user’s virtual representations providing highly accurate real-time visual cues about the surroundings users thus allowing for improved context in user interactions, applicable to the virtual rendering systems as taught in Pei and Bradski.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: See the Notice of References Cited (PTO-892)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TERRELL M ROBINSON whose telephone number is (571)270-3526. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KENT CHANG can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TERRELL M ROBINSON/Primary Examiner, Art Unit 2614
Prosecution Insights