Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. GB2402916.7, filed on 02/29/2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/06/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted on 02/13/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
The abstract of the disclosure is objected to because it repeats information given in the title. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-5 and 12-14 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Anderegg (US 11628374 B2).
Regarding claim 1, Anderegg teaches a system for animating a first virtual element within a virtual environment (col. 8, lines 31-34: “By way of example, virtual environment 236a may be a virtual field of snow for an animation in which the virtual character corresponding to selected object 122 builds a snowman.”), comprising:
receiving circuitry configured to receive first state data descriptive of a state of the first virtual element, the first state data comprising one or more kinematic properties of the first virtual element (col. 12, lines 57-63: “In one implementation, animation 132/232 of selected object 122 may be generated based on seven basic movements or postures including standing idly, walking, running, idly crouching, walking crouched, get-up, and mid-jump. Animation 132/232 of selected object 122 may be generated through blending of those movements, the default ragdoll state, and inverse kinematics.”);
generating circuitry comprising a generating model trained to generate, based on the received first state data, animation data to be applied to the first virtual element (col. 12, lines 49-56: “Flowchart 340 continues with generating animation 132/232 of selected object 122 using the determined distance 124, the movement identified in action 347, and in some implementations also using virtual environment 236a or 236b selected as a result of optional action 345 (action 348). Animation 132/232 may be generated by object animation software code 212, executed by hardware processor 214, as described in greater detail below.”); and
animating circuitry configured to apply the generated animation data to the first virtual element (col. 15, lines 51-64: “As a portable device that generates an animation receives an animation input, such as being moved in physical space or by receiving a voice or touchscreen input, movements can be transferred directly and immediately to the virtual character animating a selected object. Because the virtual character can either be visualized in the real-world using AR or visualized substantially entirely in a virtual world using VR, the experience is one of having direct control over the virtual character. However, because the character is a virtual depiction of the physical toy or other object selected by the user, the present virtual puppeteering solution can advantageously apply sophisticated animations and other logic to enhance the user's enjoyment and perception of interactivity.”).
Regarding claim 2, Anderegg teaches the system of claim 1, wherein:
the receiving circuitry is configured to receive second state data descriptive of respective states of one or more second virtual elements, the second state data comprising a respective surface geometry of each of the one or more second virtual elements (col. 12, lines 63-66: “Referring to FIG. 6A, for example, FIG. 6A shows animation 632 in which virtual character 660 corresponding to selected object 122 is animated to roll snowball 670a.”); and
the generating model is trained to generate, based on the received first state data and the received second state data, the animation data to be applied to the first virtual element (col. 13, lines 7-11: “The animation input provided by user 122 can include the position, velocity, and orientation of portable device 110/210 in space. Additionally, touchscreen display screen 118/218 can be used as a single button to select virtual object 122 depicted in animation 132/232/632.”).
Regarding claim 3, Anderegg teaches the system of claim 1, comprising first determining circuitry configured to determine, based on the received first state data and the received second state data where applicable, whether the animation data is to be generated (col. 10, lines 26-32: “One suitable default state would be to turn the virtual character into a ragdoll. Such a default state could be used when a virtual character having no jump momentum is held in the air: instead of having a character with an inappropriate jumping animation in the air, user 122 would instead be holding a virtually simulated ragdoll.”).
Regarding claim 4, Anderegg teaches the system of claim 3, comprising transmission circuitry configured to transmit the received first state data and the received second state data where applicable to a physics engine for modelling a subsequent state of the first virtual element if the first determining circuitry determines that the animation data is not to be generated (col. 10, lines 26-32, as above. NOTE: converting a virtual character into a ragdoll requires the use of a physics engine for realistic motion without the use of animation data.).
Regarding claim 5, Anderegg teaches the system of claim 1, wherein:
the receiving circuitry is configured to receive one or more performance metrics indicative of a capability with which a physics engine is to model, based on the received first state data and the received second state data where applicable, a subsequent state of the first virtual element (col. 9, lines 45-52: “In some such use cases, the fixed distance constraint can be relaxed, especially to handle collisions. For example, if animation 132 of selected object 122 would appear to push the virtual character corresponding to selected object 122 into a floor or wall, distance 124 may be shortened appropriately in order to prevent the virtual character depicting selected object 122 from appearing to phase through another solid object.”);
the system comprises second determining circuitry configured to determine, based on one or more of the received performance metrics, whether the subsequent state of the first virtual element is to be modelled by the physics engine (col. 9, lines 45-52, as above); and
the generating model is trained to generate the animation data to be applied to the first virtual element if the second determining circuitry determines that the subsequent state of the first virtual element is not to be modelled by the physics engine (col. 12, lines 49-53: “Flowchart 340 continues with generating animation 132/232 of selected object 122 using the determined distance 124, the movement identified in action 347, and in some implementations also using virtual environment 236a or 236b selected as a result of optional action 345 (action 348).”).
Regarding claim 12, Anderegg teaches the system of claim 1, wherein a given virtual element is one of:
a virtual object (col. 10, lines 39-42: “For example, when the virtual character picks up an object, its hands are moved close to the object independently of the underlying identified movement for animating the virtual object.”);
a virtual character (col. 10, lines 39-42, as above); and
at least a part of the virtual environment itself (col. 5, lines 60-64: “As further shown in FIG. 2, memory 216 contains object animation software code 212 and may optionally include virtual environment database 234 storing multiple virtual environments represented by exemplary virtual environments 236a and 236b.”).
Claim 13 is substantially similar to claim 1, and differs primarily in that it teaches a method rather than a system. It is therefore rejected on a similar basis to claim 1.
Claim 14 is substantially similar to claim 1, and differs primarily in that it teaches a storage medium rather than a system. It is therefore rejected on a similar basis to claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderegg (US 11628374 B2) as applied to claims 1-5 above, and further in view of Daviet (US 11210833 B1).
Regarding claim 6, Anderegg teaches the system of claim 5, but fails to teach wherein the one or more performance metrics comprises one or more of:
a resolution at which the virtual environment is being rendered for display;
a frame rate at which the virtual environment is being rendered for display;
an input lag between receipt of a user input signal and performance of an action within the virtual environment by an in-game avatar in response to the user input signal;
a temperature of processing circuitry configured to execute the physics engine;
an amount of electrical power consumed by processing circuitry configured to execute the physics engine;
a processing load of the physics engine; and
an amount of electrical power consumed by a cooling system configured to cool processing circuitry that is configured to execute the physics engine.
Daviet teaches one or more performance metrics (col. 4, lines 9-11: “FIG. 8 illustrates performance metrics of one embodiment compared to other methods, according to some embodiments of the present disclosure.”), comprising one or more of:
a resolution at which the virtual environment is being rendered for display (col. 31, lines 11-15: “Dimensions of such a two-dimensional array of pixel color values might correspond to a preferred and/or standard display scheme, such as 1920-pixel columns by 1280-pixel rows or 4096-pixel columns by 2160-pixel rows, or some other resolution.”);
a frame rate at which the virtual environment is being rendered for display (col. 31, lines 40-44: “A frame rate might be used to describe how many frames of the stored video sequence are displayed per unit time. Example video sequences might include 24 frames per second (24 FPS), 50 FPS, 140 FPS, or other frame rates.”);
an input lag between receipt of a user input signal and performance of an action within the virtual environment by an in-game avatar in response to the user input signal;
a temperature of processing circuitry configured to execute the physics engine;
an amount of electrical power consumed by processing circuitry configured to execute the physics engine;
a processing load of the physics engine; and
an amount of electrical power consumed by a cooling system configured to cool processing circuitry that is configured to execute the physics engine (NOTE: elements iii – vii are not taught by Daviet. However, as claim 6 teaches “one or more” of the above list of performance metrics, teaching elements i and ii is considered sufficient for the purposes of this notice.).
It would have been obvious to one familiar in the art to include the performance metrics of Daviet in the system of Anderegg, as both are in the same field of endeavor of image data processing or generation for animation. Tracking performance metrics such as resolution and frame rate is an obviously beneficial practice for an animation system to track in an interactive virtual environment, and is well-known and commonly-used in the art.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderegg (US 11628374 B2) as applied to claims 1-5 above, and further in view of Spivack (US 20190102946 A1).
Regarding claim 7, Anderegg teaches the system of claim 1, but fails to teach wherein:
the receiving circuitry is configured to receive object metadata indicating a type of object being represented by the first virtual element; and
the generating model is trained to generate at least part of the animation data based on the object metadata.
Spivak teaches receiving circuitry configured to receive object metadata indicating a type of object being represented by the first virtual element (par. 0069: “The metadata repository 124 is able to store virtual object metadata of data fields, identification of VOB classes, virtual object ontologies, virtual object taxonomies, etc.”); and
a generating model trained to generate at least part of animation data based on the object metadata (par. 0069: “One embodiment further includes the state information repository 132 which can store state data, or state metadata, or state information relating to various animation states of a given VOB or a group of VOBs. The state information repository 132 can store identifications of the number of states associated with any VOB, metadata regarding animation details of each given animation state, and/or rendering metadata of each given animation state for any VOB for the host server 100 or client device 102A-N to render, create or generate the VOBs and their associated animations in different animation states.”).
It would have been obvious to one familiar in the art to utilize the object categorization of Spivak to inform the animation system of Anderegg, as both are in the same field of endeavor of image data processing or generation for 3D animation. Using object category metadata would allow for faster processing times and more efficient animation selection, and such metadata is well-known and commonly-used in the art.
Claim(s) 8-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anderegg (US 11628374 B2) as applied to claims 1-5 above, and further in view of Ackerson (US 20230281955 A1).
Regarding claim 8, Anderegg teaches the system of claim 1, but fails to teach wherein the generating model is trained using one or more sets of image data of one or more real-world objects.
Ackerson teaches a generating model trained using one or more sets of image data of one or more real-world objects (par. 0188: “At the start, any specified point and direction may return meaningless values. The exemplary system may then be trained using calibrated images from various known viewpoints (e.g., a few hundred images from random locations on a hemisphere above a real or simulated scene) or other image-related information. In some embodiments, the process may be initiated by selecting one or a set of pixels in the training images.”).
It would have been obvious to one familiar in the art to utilize the training dataset of Ackerson to train the animation system of Anderegg, as both are in the same field of endeavor of image data processing or generation for 3D modeling for computer graphics. Using one or more sets of image data for training a generating model would allow for more accurate model generation, and such training datasets are well-known and commonly-used in the art.
Regarding claim 9, Anderegg and Ackerson teach the system of claim 8. Ackerson further teaches wherein:
each set of image data comprises a plurality of image frames (par. 0034: “The 3D label can correspond to all of the key-frames collected during mapping. The 3D label can include a bounding region superimposed over a display of the introduced object and text indicative of the introduced object. In other words, a label corresponding to the point cloud representative of the introduced object can be applied to those pixels representative of the introduced object in each frame of a video stream. The saved key frame labels and associated images can be used as annotations and corresponding training images to train a ML model.”) and a set of training kinematic properties (par. 0057: “FIG. 6 shows an illustrative example of a system 600 that provides object location using machine learning and SLAM components, in an embodiment.”); and
the generating circuitry is configured to:
analyse each set of image data to identify a real-world object (par. 0058: “Once trained, in an embodiment, the system is now able to identify objects and where they are in the environment around the robot.”),
determine inter-frame motion of the identified real-world object, the inter-frame motion comprising a sequence of changes in position and/or changes in orientation of the identified real-world object between successive image frames of the set of image data (par. 0051: “In some embodiments, the invention may be configured to create a model of a scene using static data (i.e., data captured of the scene where the contents of the scene are not moving) or a dynamic scene (i.e., data captured of the scene where the contents of the scene are moving relative to each other and/or the image capture device). Similarly, the model may be configured to represent a scene, portion of a scene, or one or more objects in a scene in a static configuration (i.e., the reconstruction depicts the scene where the contents of the scene are not moving) or a dynamic configuration (i.e., where a portion of or all of the contents of the scene are in motion).”), and
provide the inter-frame motion and the set of training kinematic properties to the generating model for the generating model to learn a correlation therebetween (par. 0052: “The invention described herein may provide advantages over conventional representations of dynamic scenes. For example, in some known systems for representing a scene (e.g., where the representation primarily regards the scene's light field rather than the scene's matter field), there may be challenges representing dynamism because the associated light characteristics are directly associated with media in the scene, causing a need to reinitialize and/or retrain large portions of the scene model for every time step where the matter field has changed configuration (e.g., changed shape or movement). In some embodiments hereof, when reconstructing a dynamic scene, the inventions described herein may calculate the interaction with the light field in the scene with the portions of the scene in motion, allowing for better understanding of the media comprising such objects.” NOTE: in order to calculate the interaction with the light field in the scene with the portions of the scene in motion, the system would necessarily have to be able to learn a correlation between inter-frame motion and kinematic properties.).
The motivation to combine Ackerson with Anderegg is substantially similar to that discussed in the rejection of claim 8 above.
Regarding claim 10, Anderegg teaches the system of claim 1, but fails to teach wherein the generating model is trained using one or more datasets of one or more virtual elements.
Ackerson teaches a generating model trained using one or more datasets of one or more virtual elements (par. 0200: “By obtaining 3D models of real-world objects with included BLIF information, 2D training datasets may be synthetically generated by rendering the models from various viewpoints and/or varying the lighting and BLIF parameters appropriately. In such embodiments, the system may be used to provide a vast number of training or synthetic datasets to the TMLM.”).
It would have been obvious to one familiar in the art to utilize the training dataset of Ackerson to train the animation system of Anderegg, as both are in the same field of endeavor of image data processing or generation for 3D modeling for computer graphics. Using one or more sets of image data for training a generating model would allow for more accurate model generation, and such training datasets are well-known and commonly-used in the art.
Regarding claim 11, Anderegg teaches the system of claim 1, but fails to teach wherein the generating model is trained to generate the animation data by:
generating a latent space based on training data input thereto; and
selecting, based on the received first state data, one or more latent variables from the generated latent space, each latent variable being associated with a kinematic property of the virtual element.
Ackerson teaches generating model is trained to generate the animation data by:
generating a latent space based on training data input thereto (par. 0197: “Some embodiments of the invention may implement NeRF Self Supervised Object Segmentation (NeRF SOS) or an analogous processing regime to use a latent representation for downstream object recognition, object segmentation, and/or other tasks. In some embodiments, RMF data may be used as a latent representation of the scene and used for downstream tasks, such as object recognition and/or segmentation.”); and
selecting, based on the received first state data, one or more latent variables from the generated latent space, each latent variable being associated with a kinematic property of the virtual element (par. 0216: “The system may be configured to generate high-resolution images with fine-grained control over various aspects of the image, such as the pose, expression, and appearance of the subject. For example, some embodiments of the invention may be configured to use StyleGANs configured to use a “style” vector. Such a vector may control the various properties of the generated image, may be learned during the training process, and may be manipulated to generate new images with different styles to enrich generation capacity to relight scenes. In similar ways, the current system may be configured to generate latent variables for light and material properties which can help improve deconstructability and/or reconstructability of a scene.”).
It would have been obvious to one familiar in the art to utilize the latent variables of Ackerson in the animation system of Anderegg, as both are in the same field of endeavor of image data processing or generation for 3D modeling for computer graphics. Using latent variables would allow for more accurate model generation, and such variables are well-known and commonly-used in the art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN A BARHAM whose telephone number is (571)272-4338. The examiner can normally be reached Mon-Fri, 8:30am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu, can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN ALLEN BARHAM/Examiner, Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights