DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-18 are currently pending in the present application, with claims 1 and 10 being independent.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/03/2025 has been considered by the examiner.
Claim Objections
Claims 1, 11-12, and 16-17 is/are objected to because of the following informalities:
The recitation of “comprises a server and one or more terminal devices…” in claim 1, line 3 should be amended to “comprises: “including a colon after the word “comprises” to separate the preamble from the body of the claim.
The recitation of “may include” in claims 11-12 is exemplary language and does not positively recite a claim limitation. Examiner suggests amending the claims to remove the permissive language or otherwise positively recite the intended subject matter, rather than including optional limitations.
The recitation of “a step of analyzing user data, in which the real-time analysis module performs an emotional analysis on the collected real-time user image data, real- time user audio data, and/or user motion data” in claim 16 is recited twice, and should be amended to remove the repeated recitation.
The recitation of "a step of collecting environmental data, in which, in which the real-time analysis module collects environmental change data from the virtual environment" in claim 17 should be amended to "a step of collecting environmental data,
Appropriate correction is required.
Claim 12 is objected to under 37 CFR 1.75 as being a substantial duplicate of claim 11. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a data processing module…configured to analyze…” in claim 1.
“a virtual pet generation module…configured to generate…” in claim 1.
“a real-time analysis module…configured to receive…” in claim 1.
“a motion sensing module for capturing…” in claims 3, and 11-12.
“an environment generation module for generating…” in claim 4.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
a data processing module: corresponding structure described in the specification in Par. [20], [27], [33-40], Fig. 4-5.
a virtual pet generation module: corresponding structure described in the specification in Par. [20], [28].
a real-time analysis module: corresponding structure described in the specification in Par. [20], [23], [30], [41]-60], Fig. 6, and 8-11.
environment generation module: corresponding structure described in the specification in Par. [20], [29], [54], Fig. 8-9.
motion sensing module: corresponding structure described in the specification in Par. [22]-[23], [32], [41]-43].
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION. —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, claim 1 recites a virtual pet generation module configured to generate a virtual identity for the particular animal based on the specific 3D appearance mode, specific behavior model, and specific voice model of the specific animal to “complete the reconstruction.” The claims do not define the “reconstruction” or identify and objective standard or operation by which the reconstruction is considered “complete.” It is unclear whether “complete the reconstruction” is merely generating the virtual identity or virtual pet, incorporating each of the recited models into the virtual identity, or satisfying some degree of similarity to the real animal. Therefore, the scope of “complete the reconstruction” cannot be determined with reasonable certainty.
Further, claim 1 recites the limitation “cause the virtual identity to execute any one or a combination of the specific 3D appearance model, specific, behavior mode, and specific voice model…” It is unclear what is meant by a virtual identity “executing” a 3D appearance model, a behavior model, or a voice model. For example, it is unclear whether “execute” means selecting a model, processing data through the model, rendering an output from the model, or performing an action represented by the model and its relationship to the “virtual identity”. Therefore, the scope of the claimed execution operations cannot be determined with reasonable certainty.
Claims 5, 6, and 10 recites substantially similar subject matter as to that of claim 1 and is rejected using substantially similar rationale as to that which was set forth with respect to claim 1.
Additionally, claim 1 recites “adjusting various parameters…to perform a corresponding action.” It is unclear whether the action corresponds to the user command, the selected model, the adjusted parameters, or a predefined action associated with the virtual identity. Therefore, the scope of “corresponding action” cannot be determined with reasonable certainty.
Regarding claim 1, claim 1 recites the limitation "the particular animal" in line 16. There is insufficient antecedent basis for this limitation in the claim.
Claims depending thereon are also rejected for substantially similar reasons as those set forth for the claims from which they depend on.
Regarding claim 13, claim 13 recites the limitation "the model optimization step" in line 2 of the claim. There is insufficient antecedent basis for this limitation in the claim.
The examiner respectfully requests the applicant to clarify the scope of the claimed invention.
Regarding claim 16, claim 16 recites “generating a user command, in which the real-time analysis module generates a corresponding user command for the virtual identity…” It is unclear what the generated user command “corresponds” to. Specifically, the claim does not establish whether the command corresponds to the user, the collected user data, the emotional analysis result, or an intended action of the virtual identity. It is further unclear whether the corresponding user command is intended for the “specific animal,” since the method of claim 10 from which claim 16 depends thereon recites generating “a virtual identity for the specific animal.” Thus, the relationship among the user, the specific animal, the virtual identity, and the user command generated “based on the emotional analysis results” is unclear.
Claim 16 further recites” the virtual identity adjusts the various parameters…after receiving the user command based on the emotional analysis results.” It is unclear whether the virtual identity itself performs the parameter-adjustment operation or whether the real-time analysis module or server adjusts the models used to generate the virtual identity. It is further unclear whether the recited adjustment of behavior and/or emotional expression refers to adjusting the user’s behavior or emotional expression of the specific animal represented by the virtual identity. Additionally, which parameters are adjusted, as recited as “adjusts various parameters,” is unclear as to whether parameters in each of the specific recited models are to be adjusted, the user’s behavior or emotional expressions are adjusted, or the virtual identity’s. Therefore, the required relationship among the emotional analysis results, the generated user command, and the resulting parameter adjustment operations cannot be determined with reasonable certainty.
Claims 17 and 18 recites substantially similar ambiguity regarding generating “a corresponding” command for the virtual identity, and further “the virtual identity adjusts various parameters…” as to that of claim 16 and is rejected using substantially similar rationale as to that which was set forth with respect to claim 16.
Claims 1-18 will be examined as best understood by the examiner.
Regarding claim 1, claim limitation “virtual pet generation module” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. Par. [20], [28], and step S3 merely repeat that module 114 generates virtual identity 1141 from the specific appearance, behavior, and voice models. The specification does not disclose a structure or algorithm explaining how the three models are retrieved, integrated, or applied to generate an operable virtual identity or determine that reconstruction is complete. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Regarding claim 4, claim limitation “environment generation module” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. Par. [20], [29] states that module 115 generates a virtual environment and transmits an environmental command, while Par. [54] merely provides examples of resulting environmental effects. The specification does not disclose a structure or algorithm for generating the virtual environment, determining an environmental command, or transmitting an environmental command. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
---------------------------------------------------------------------------------------------------------------
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 1-18 is/are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claim 1, the originally filed disclosure does not reasonably convey to one of ordinary skill in the art that the inventor possessed the claimed “virtual pet generation module” for generating a virtual identity based on the specific 3D appearance mode, specific behavior model, and specific voice model of the specific animal to complete the reconstruction. Par. [20] and [28] of the originally filed disclosure merely repeat that the modules generate the virtual identity from the recited models, but does not disclose an algorithm, processing sequence, or other technical implementation explaining how the recited models are processed to generate the virtual identity.
Regarding claim 4, the originally filed disclosure does not reasonably convey to one of ordinary skill in the art that the inventor possessed the claimed “environment generation module” for generating a virtual environment and transmitting at least one environmental command to the real-time analysis module. Par. [20] and [29] of the originally filed disclosure merely repeat that the environment generation module generates virtual environment and transmits an environmental command. Although Par. [51]-[54] identify examples of environmental effects, the specification does not disclose an algorithm, processing sequence, or other implementation for generating the virtual environment, and generating/transmitting the corresponding environmental command.
Claims 10 recites substantially similar subject matter as to that of claim 1 and is rejected using substantially similar rationale as to that which was set forth with respect to claim 1.
Claims depending thereon are also rejected for substantially similar reasons as those set forth for the claims from which they depend on.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lim (KR 20230103666 A), in view of Xu et al. (CN 112785681 A), hereinafter referred to as “Xu”.
Regarding claim 1, Lim discloses a pet reconstruction system based on a virtual world infrastructure, wherein the system is built on a virtual world infrastructure and primarily comprises a server and one or more terminal devices (Fig. 1 and Par. 0029-0031; The user terminal 100…The server 200…operated by a metaverse virtual reality platform-based online service provider…The wired/wireless communication network 300…), the server comprising:
a virtual pet generation module, in data communication with the server, configured to generate a virtual identity for the particular animal based on the specific 3D appearance model, specific behavior model, and specific voice model of the specific animal to complete the reconstruction (Fig. 4-8 and Par. 0030; The server 200 generates a virtual reality animal avatar having the same or similar appearance, behavioral pattern, and personality to a real companion animal based on the characteristic information of the actual companion animal provided by the user terminal 100 (100-1, 100-2, ..., 100-n) through the wired/wireless communication network 300, trains or modifies the behavioral pattern and personality of the virtual reality animal avatar according to the characteristic information or feedback information input from the user, and based on the generated appearance, behavioral pattern, and personality. Par. 0063; The finally created companion animal avatar can operate in the virtual world in a 3D form); and
a real-time analysis module, in data communication with the server, configured to receive at least one user command from the terminal device and cause the virtual identity to execute any one or a combination of the specific 3D appearance model (Fig. 5 and Par. 0056-0063; a process for generating a companion animal avatar in virtual reality similar in appearance and personality to a real companion animal…the user's terminal can start creating a companion animal avatar by taking a photograph of an actual companion animal using a camera in the terminal. The terminal can identify the type of companion animal from the captured image…The terminal may perform detailed modifications to the companion animal avatar by additionally reflecting detailed appearance characteristics. For example, information about hair color, eye size and distance, nose size, etc. may be additionally reflected by receiving input from the user), specific behavior model (Fig. 6-7 and Par. 0065-0090; a process for forming a companion animal in virtual reality having behavior patterns and personalities similar to those of an actual user's companion animal…Movement information based on personality information including the currently formed behavioral pattern of the companion animal avatar for each situation is transmitted to the user terminal from the server along with a simulation of an arbitrary situation...The user can see how the companion animal avatar behaves in the corresponding situation through the output device in the terminal and input positive (Buy in) or negative (Buy out) feedback…The user may go beyond inputting feedback information and more actively create a simulation of a situation in which the companion animal avatar will operate…), and specific voice model according to the user command (Fig. 8 and Par. 0092-0114; a process for learning so that a companion animal avatar in virtual reality can communicate with a user by voice, assuming what emotion and how to communicate with a person based on the behavior pattern and personality formed through the above process…the companion animal avatar can have a conversation with the user, if the user's actual companion animal avatar has a personality, it provides feedback on how to respond in a specific conversation situation, thereby providing a basis for the server to learn in which direction the conversation will proceed when the companion animal avatar has a later conversation…the server may present several example answers to the terminal and suggest to the user to select possible answers from among the answers in order of priority. That is, the user provides feedback on how to respond in the case of the user's actual companion animal avatar's personality in a specific conversation situation…), thereby adjusting various parameters in the specific 3D appearance model (Par. 0062-0063; Generation of the companion animal avatar can be completed by receiving a confirmation input based on a comparison of the appearance of the companion animal created by the user and the actual companion animal at each stage of the companion animal avatar creation process…created companion animal avatar can operate in the virtual world in a 3D form), specific behavior model (Par. 0070-0074; User can input positive or negative feedback about this. Feedback input by the user may be the basis for determining how the server will behave in a similar situation with respect to the companion animal avatar in the future…In a specific situation set by the user, by seeing how the current companion animal behaves based on the preset personality information, the user can provide additional feedback and gradually learn the companion animal avatar to be similar to the actual companion animal…Feedback input from the user may be reflected to the companion animal avatar by the server after the simulation is finished…), and specific voice model to perform a corresponding action (Par. 0102-0114; Based on the completed conversation set, the server can learn in which direction the conversation will proceed when the companion animal avatar has a conversation later…The companion animal avatar's answer may be selected according to the chatbot's expected answer of the companion animal selected by the user through positive (buy-in) or negative (buy-out) feedback. The server can learn based on the expected answer selected by the user…Answers of companion animal avatars generated by the server may be output as voices in text to speech (TTS) from the terminal. Through this, users can have the experience of having a conversation with a companion animal avatar in the metaverse virtual reality).
Lim does not disclose a model database, in data communication with the server, configured to store multiple 3D appearance models, multiple behavior models, and multiple voice models for various animal species; a customization database, in data communication with the server, configured to store uploaded specific appearance features, specific behavior features, and specific voice features of a specific animal; a data processing module, in data communication with the server, configured to analyze the specific appearance features, specific behavior features, and specific voice features of the specific animal to generate a specific 3D appearance model, a specific behavior model, and a specific voice model for the specific animal, and store the generated models in the customization database;
In the same art of generating and modifying 3D virtual pets, Xu discloses a model database, (Par. 0093-0095; …calling the standard 3D model corresponding to the recognized category from a pre-established standard 3D model database, wherein how many are stored in the standard 3D model database Standard 3D model corresponding to each pet category. The standard 3D model stored in the standard 3D model database may be established based on a large number of sample pictures or video training of each pet, or may be a 3D model of various pets set manually), multiple behavior models (Par. 0110; the pet can be called from a pre-established standard 3D model database The action 3D model corresponding to the category of the pet, wherein the standard 3D model database stores multiple action 3D models corresponding to the pet category), and multiple voice models for various animal species (Par. 0114; call the action 3D model corresponding to the type of pet and the corresponding sound data from a pre-established standard 3D model database, wherein the standard 3D model database The action 3D models and corresponding sound data corresponding to multiple pet types are stored in it…);
a customization database, (Par. 0091-0097; Acquire a pet picture or video taken by the user…each video frame is extracted from the video, and the video frame is input into the corresponding recognition model for recognition in the subsequent steps…Acquire biometric data of the pet from the pet picture or video…the biological characteristics of the pet may include the pet's size, facial characteristics, skin and hair characteristics, limb characteristics, and/or tail characteristics. In practical applications, the above-mentioned biometric data of the pet can be obtained from the pet picture or video through the feature extraction technology…), specific behavior features (Par. 0103-0110; the facial expression recognition is performed on the pet pictures or videos taken by the user, the pictures or video frames of the corresponding facial expressions can be marked with facial expression names, such as happy, scared, sad, angry, etc. The pet pictures or video frames taken by the user can be stored in the database after the emoticon name is marked for subsequent use…multiple continuously shot pictures or video frames recognized as the same facial expression may also be combined into the action corresponding to the facial expression picture collection…), and specific voice features of a specific animal (Par. 0111-0113; …when obtaining a video of a user shooting a pet, the sound collected during shooting can also be obtained, and the sound corresponding to each video frame in the action picture set corresponding to each expression is merged and saved in the order of the video frame to obtain each action picture Set the corresponding sound data…);
a data processing module, configured to analyze the specific appearance features (Par. 0091-100; Recognize the type of pet in the pet picture or video through the pre-trained pet type recognition model, and call the corresponding standard 3D model according to the recognized type…acquiring multiple pet picture samples for each pet to form a training sample set; for each pet picture sample in the training sample set Annotation processing is performed to annotate the type of pet in each pet image sample; the neural network is trained through the training sample set after the annotation processing to obtain a pet type recognition model. The pet picture taken by the user or the video frame in the video is input into the pet type recognition model, and the pet type recognition model can output the pet type after the recognition processing…Acquire biometric data of the pet from the pet picture or video, and combine the biometric data with the standard 3D model to generate a 3D model of the pet…), specific behavior features (Par. 0107-0110; after performing facial expression recognition on the pet picture or video through the pre-established facial expression recognition model in step S4, multiple continuously shot pictures or video frames recognized as the same facial expression may also be combined into the action corresponding to the facial expression Picture collection. For example, if the user takes 10 pet pictures in a row, the pictures 1-5 are recognized as expression A, and pictures 6-10 are recognized as expression B, then the pictures 1-5 are combined into the action corresponding to expression A Picture set 1, combine the 6th-10th pictures into the action picture set 2 corresponding to the expression B…the action picture set corresponding to the expression set by the user can be obtained, and multiple pictures in the obtained action picture set can be synthesized into the 3D model of the pet to generate the pet A 3D action image with the expression), and specific voice features of the specific animal (Par. 0111-0114; …when obtaining a video of a user shooting a pet, the sound collected during shooting can also be obtained, and the sound corresponding to each video frame in the action picture set corresponding to each expression is merged and saved in the order of the video frame to obtain each action picture Set the corresponding sound data. For example, if the user takes a 10-second video, the video frame from 1-5 seconds is recognized as expression A, and the video frame from 6-10 seconds is recognized as expression B, and the video frames from 1-5 seconds are recognized as expression B. The frame is combined into action picture set 1 corresponding to expression A, and the collected sounds of 1-5 seconds are saved as sound data corresponding to action set 1, and the video frames of 6-10 seconds are combined into the action picture corresponding to expression B Set 2, at the same time, save the collected sounds of the 6th-10th second as the sound data corresponding to action set 2…) to generate a specific 3D appearance model, a specific behavior model, and a specific voice model for the specific animal (Fig. 1-2 and Par. 0096; combine the biometric data with the standard 3D model to generate a 3D model of the pet…Par. 0101; In the 3D model of the pet, a 3D image of the pet with the expression is generated…Par. 0109; the obtained action picture set can be synthesized into the 3D model of the pet to generate the pet A 3D action image with the expression…Par. 0111; when generating a 3D action image of the pet with the expression, corresponding sounds, such as a call, can be used to generate the 3D image with a voice of the pet with the expression…Par. 0113; after the user sets the pet expression, the action picture set corresponding to the expression set by the user and the corresponding sound data can be obtained, and the video frames in the obtained action picture set can be synthesized into the 3D model of the pet), and store the generated models in the customization database (Par. 0129; The standard 3D model corresponding to the identified category is called from a pre-established standard 3D model database, wherein the standard 3D model database stores multiple standard 3D models corresponding to pet categories. Par. 0138; Calling the action 3D model corresponding to the pet category from a pre-established standard 3D model database, wherein the standard 3D model database stores multiple action 3D models corresponding to the pet category. Par. 0148; Call the action 3D model and corresponding sound data corresponding to the type of pet from a pre-established standard 3D model database, wherein the standard 3D model database stores action 3D models and corresponding sound data corresponding to multiple pet types).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Xu’s model and customization database into Lim’s server-based virtual pet system. The motivation lies in the advantage of centralizing and maintaining the information used to generate and update the virtual pet, allows efficient retrieval of a relevant base model, and enable reuse of existing appearance, action, and sound information data gathered rather than reprocessing the data gathered whenever the virtual pet is generated, accessed, or refined. Further, separately maintaining specified models and their associated data would further allow efficient organization across multiple virtual pets while preserving the distinguishing characteristics of each particular animal. This organization would have constituted a predictable use of databases for their established functions of storing, organizing, retrieving, reusing, and updating information, yielding predictable results in reduced repetitive processing and storage and improving the overall efficiency and consistency of model generation and modification in virtual avatar/animal generation.
Regarding claim 2, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 1, and Lim further discloses wherein the terminal device is in data communication with the server via the Internet for downloading the virtual identity, which includes the specific 3D appearance model, specific behavior model, and specific voice model of the specific animal (Fig. 1-3 and Par. 0029-0040; The wired/wireless communication network 300 provides a communication path through which the user terminals 100: 100-1, 100-2, ..., 100-n, and the server 200 can transmit and receive signals and data with each other…when configured as an Internet protocol (IP) based system, the wired/wireless communication network 300 may be implemented as a wired/wireless Internet network, and when the user terminals 100 (100: 100-1, 100-2, ..., 100-n) and the server 200 are implemented as mobile communication terminals, the wired/wireless communication network 300 may be implemented as a cellular network or a wireless network such as a wireless local area network (WLAN) network…).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 3, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 1, and Lim further discloses wherein the terminal device further comprises:
a video camera for capturing real-time image data from the real world (Fig. 2; camera 160. Par. 0040; The camera 160 may be connected to the processor 130 and capture an image of a front object. Fig. 5 and Par. 0057; the user's terminal can start creating a companion animal avatar by taking a photograph of an actual companion animal using a camera in the terminal); and
a microphone for capturing real-time audio data from the real world (Par. 0140; the input device may include a microphone. The method may further include transmitting a voice message for the user's companion animal avatar acquired through the microphone to the server, receiving a reply message of the companion animal avatar generated based on the voice message and the personality information from the server, and outputting the reply message in video and audio form through the output device as the companion animal avatar speaks), wherein the video camera includes a motion sensing module for capturing motion data of at least one object within the real-time image data and transmitting the motion data to the server (Par. 0015; The terminal includes a transceiver, a memory, a processor, an input device, an output device, and a camera. The method includes a process of transmitting an image of a companion animal acquired using the camera to a server, a process of receiving first motion information about a motion of a companion animal avatar generated based on the image from the server in a virtual space, a process of outputting a motion of the companion animal avatar in the virtual space through the output device based on the first motion information…).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 4, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 3, and Lim further discloses wherein the server further comprises an environment generation module for generating a virtual environment and transmitting at least one environmental command to the real-time analysis module (Par. 0072; The user may go beyond inputting feedback information and more actively create a simulation of a situation in which the companion animal avatar will operate. In a specific situation set by the user…Par. 0080; companion animal avatar on a walk…).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 5, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 4, and Lim further discloses wherein the real-time analysis module receives the environmental command from the environment generation module and causes the virtual identity to execute any one or a combination of the specific 3D appearance model, specific behavior model, and specific voice model according to the environmental command (Fig. 6-7, Par. 0067-0070 and Par. 0078-0081; a simulation of an arbitrary situation…Par. 0087-0090; The companion animal's appearance, physical conditions, behavioral characteristics, and personality characteristics learned so far are accumulated in the companion animal avatar, and when a specific situation occurs within the metaverse virtual reality, the companion animal avatar reflects the characteristics and acts…Par. 0139; include receiving, along with the first action information, arbitrary simulation information about a first situation in which the companion animal avatar is located from the server. The feedback information may include a first input or a second input for a motion based on the first motion information of the companion animal avatar in the first situation. When simulation information of a second situation similar to the first situation is received together with the second motion information, the companion animal avatar may operate according to the second motion information generated based on the feedback information).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 6, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 4, and Lim further discloses wherein the real-time analysis module further enables the virtual identity to send and/or receive at least one virtual animal command from at least one other virtual identity in the virtual environment and causes the virtual identity to execute any one or a combination of the specific 3D appearance model, specific behavior model, and specific voice model according to the virtual animal command (Par. 0080-0089; given a situation where you meet a dog that is bigger than your companion animal avatar on a walk, the companion animal avatar can perform one or more of the following actions: 1) turn a blind eye, 2) bark aggressively, and 3) wonder and greet you and greet your scent…User can input positive or negative feedback about this. Feedback input by the user may be the basis for determining how the server will behave in a similar situation with respect to the companion animal avatar in the future…Par. 0123-0132; When a companion animal avatar created in the metaverse virtual reality comes into contact with other companion animal avatars, the preset behavior and personality characteristics of each avatar are internalized and interacted with…a function of automatically selecting and providing content related to companion animal training, education, and care specialized for a corresponding group may be provided).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 7, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 4, and Lim further discloses wherein each 3D appearance model and specific 3D appearance model respectively include at least one or a combination of the following parameters: breed (Par. 0059; The terminal may identify the detailed species of the companion animal based on the determined type of companion animal…), gender, age, skin color, fur color, markings, body size, and physical structure (Par. 0061; The terminal may perform detailed modifications to the companion animal avatar by additionally reflecting detailed appearance characteristics. For example, information about hair color, eye size and distance, nose size, etc.);
wherein each behavior model and specific behavior model respectively include at least one or a combination of the following parameters: sleeping, foraging, communicating, entertaining, learning, attacking, and defending (Par. 0069; the companion animal avatar can perform one or more of the following actions: 1) eat everything even when full, 2) serve food autonomously, and 3) eat as much as you can eat and keep others from eating. Par. 0080; given a situation where you meet a dog that is bigger than your companion animal avatar on a walk, the companion animal avatar can perform one or more of the following actions: 1) turn a blind eye, 2) bark aggressively, and 3) wonder and greet you and greet your scent. Par. 0088-0090; For example, in personality traits, an aggressive dog that barks furiously when meeting a new dog will behave that way if it happens to encounter another dog within the metaverse. For example, in personality traits, when the owner goes out, that is, when the user does not access the companion animal application in the terminal, the dog that waits only at the door waits with the corresponding behavior pattern while the owner does not access the app. For example, in behavioral characteristics, a dog that continues to eat snacks even when full will get sick from overeating item snacks);
and wherein each voice model and specific voice model respectively include at least one or a combination of the following parameters: pitch, frequency, volume, rhythm, continuity, and voice type (Par. 0109; I can complete a set that configures the chatbot scenario to suit my pet. In addition, the voice of the companion animal can be voiced with the voice tone selected by the user).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 8, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 7, and Lim further discloses wherein the real-time analysis module further adjusts the parameters in the specific 3D appearance model, specific behavior model, and specific voice model according to the environmental command (Par. 0062-0063; Generation of the companion animal avatar can be completed by receiving a confirmation input based on a comparison of the appearance of the companion animal created by the user and the actual companion animal at each stage of the companion animal avatar creation process…created companion animal avatar can operate in the virtual world in a 3D form. Par. 0070-0074; User can input positive or negative feedback about this. Feedback input by the user may be the basis for determining how the server will behave in a similar situation with respect to the companion animal avatar in the future…In a specific situation set by the user, by seeing how the current companion animal behaves based on the preset personality information, the user can provide additional feedback and gradually learn the companion animal avatar to be similar to the actual companion animal…Feedback input from the user may be reflected to the companion animal avatar by the server after the simulation is finished…Par. 0102-0114; Based on the completed conversation set, the server can learn in which direction the conversation will proceed when the companion animal avatar has a conversation later…The companion animal avatar's answer may be selected according to the chatbot's expected answer of the companion animal selected by the user through positive (buy-in) or negative (buy-out) feedback. The server can learn based on the expected answer selected by the user…Answers of companion animal avatars generated by the server may be output as voices in text to speech (TTS) from the terminal. Through this, users can have the experience of having a conversation with a companion animal avatar in the metaverse virtual reality).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 9, Lim in view of Xu discloses the pet reconstruction system based on a virtual world infrastructure as claimed in claim 7, and Lim further discloses wherein the real-time analysis module further adjusts the parameters in the specific 3D appearance model, specific behavior model, and specific voice model based on the real-time image data transmitted from the video camera, real-time audio data transmitted from the microphone, and/or motion data transmitted from the motion sensing module to the server (Par. 0135-0140; The terminal includes a transceiver, a memory, a processor, an input device, an output device, and a camera. The method includes a process of transmitting an image of a companion animal obtained using the camera to a server…the process of transmitting the feedback information of the first input or the second input for the first motion information to the server, the process of receiving the second motion information of the companion animal avatar generated based on the feedback information, and the process of outputting the motion of the companion animal avatar in the virtual space through the output device based on the second motion information…The method may further include transmitting a voice message for the user's companion animal avatar acquired through the microphone to the server, receiving a reply message of the companion animal avatar generated based on the voice message and the personality information from the server).
Lim and Xu are combined for the reasons set forth above with respect to claim 1.
Regarding claim 10, Lim discloses a method for implementing a pet reconstruction system based on a virtual world infrastructure (Fig. 1 and Par. 0029-0031; The user terminal 100…The server 200…operated by a metaverse virtual reality platform-based online service provider…The wired/wireless communication network 300…), comprising:
a virtual pet generation step, in which a virtual pet generation module of the server generates a virtual identity for the specific animal based on the specific 3D appearance model, specific behavior model, and specific voice model (Fig. 4-8 and Par. 0030; The server 200 generates a virtual reality animal avatar having the same or similar appearance, behavioral pattern, and personality to a real companion animal based on the characteristic information of the actual companion animal provided by the user terminal 100 (100-1, 100-2, ..., 100-n) through the wired/wireless communication network 300, trains or modifies the behavioral pattern and personality of the virtual reality animal avatar according to the characteristic information or feedback information input from the user, and based on the generated appearance, behavioral pattern, and personality. Par. 0063; The finally created companion animal avatar can operate in the virtual world in a 3D form);
an environment generation step, in which an environment generation module of the server generates a virtual environment (Par. 0051; The companion animal avatar characteristic simulation set formed through mutual information exchange between the user 100 and the server 200 creates an avatar through accumulated data in the server 200 and secures a variety of simulations and options);
and an interaction step, in which a real-time analysis module of the server receives at least one user command from one or more terminal devices in data communication with the server and/or at least one environmental command from the environment generation module, causing the virtual identity to execute any one or a combination of the specific 3D appearance model (Fig. 5 and Par. 0056-0063; a process for generating a companion animal avatar in virtual reality similar in appearance and personality to a real companion animal…the user's terminal can start creating a companion animal avatar by taking a photograph of an actual companion animal using a camera in the terminal. The terminal can identify the type of companion animal from the captured image…The terminal may perform detailed modifications to the companion animal avatar by additionally reflecting detailed appearance characteristics. For example, information about hair color, eye size and distance, nose size, etc. may be additionally reflected by receiving input from the user…Generation of the companion animal avatar can be completed by receiving a confirmation input based on a comparison of the appearance of the companion animal created by the user and the actual companion animal at each stage of the companion animal avatar creation process…created companion animal avatar can operate in the virtual world in a 3D form), specific behavior model (Fig. 6-7 and Par. 0065-0090; a process for forming a companion animal in virtual reality having behavior patterns and personalities similar to those of an actual user's companion animal…Movement information based on personality information including the currently formed behavioral pattern of the companion animal avatar for each situation is transmitted to the user terminal from the server along with a simulation of an arbitrary situation...The user can see how the companion animal avatar behaves in the corresponding situation through the output device in the terminal and input positive (Buy in) or negative (Buy out) feedback…User can input positive or negative feedback about this. Feedback input by the user may be the basis for determining how the server will behave in a similar situation with respect to the companion animal avatar in the future…The user may go beyond inputting feedback information and more actively create a simulation of a situation in which the companion animal avatar will operate. In a specific situation set by the user, by seeing how the current companion animal behaves based on the preset personality information, the user can provide additional feedback and gradually learn the companion animal avatar to be similar to the actual companion animal…Feedback input from the user may be reflected to the companion animal avatar by the server after the simulation is finished…), and specific voice model according to the user command and/or the environmental command (Fig. 8 and Par. 0092-0114; a process for learning so that a companion animal avatar in virtual reality can communicate with a user by voice, assuming what emotion and how to communicate with a person based on the behavior pattern and personality formed through the above process…the companion animal avatar can have a conversation with the user, if the user's actual companion animal avatar has a personality, it provides feedback on how to respond in a specific conversation situation, thereby providing a basis for the server to learn in which direction the conversation will proceed when the companion animal avatar has a later conversation…the server may present several example answers to the terminal and suggest to the user to select possible answers from among the answers in order of priority. That is, the user provides feedback on how to respond in the case of the user's actual companion animal avatar's personality in a specific conversation situation…Based on the completed conversation set, the server can learn in which direction the conversation will proceed when the companion animal avatar has a conversation later…The companion animal avatar's answer may be selected according to the chatbot's expected answer of the companion animal selected by the user through positive (buy-in) or negative (buy-out) feedback. The server can learn based on the expected answer selected by the user…Answers of companion animal avatars generated by the server may be output as voices in text to speech (TTS) from the terminal. Through this, users can have the experience of having a conversation with a companion animal avatar in the metaverse virtual reality).
Lim does not disclose a data upload step, in which specific appearance features, specific behavior features, and specific voice features of a specific animal are uploaded to a customization database of a server; a model construction step, in which a data processing module of the server analyzes the specific appearance features, specific behavior features, and specific voice features of the specific animal to generate a specific 3D appearance model, a specific behavior model, and a specific voice model for the specific animal, and stores the generated models in the customization database;
In the same art of generating and modifying 3D virtual pets, Xu discloses a data upload step, in which specific appearance features (Par. 0091-0097; Acquire a pet picture or video taken by the user…each video frame is extracted from the video, and the video frame is input into the corresponding recognition model for recognition in the subsequent steps…Acquire biometric data of the pet from the pet picture or video…the biological characteristics of the pet may include the pet's size, facial characteristics, skin and hair characteristics, limb characteristics, and/or tail characteristics. In practical applications, the above-mentioned biometric data of the pet can be obtained from the pet picture or video through the feature extraction technology…), specific behavior features (Par. 0103-0110; the facial expression recognition is performed on the pet pictures or videos taken by the user, the pictures or video frames of the corresponding facial expressions can be marked with facial expression names, such as happy, scared, sad, angry, etc. The pet pictures or video frames taken by the user can be stored in the database after the emoticon name is marked for subsequent use…multiple continuously shot pictures or video frames recognized as the same facial expression may also be combined into the action corresponding to the facial expression picture collection…), and specific voice features of a specific animal are uploaded to a customization database (Par. 0111-0113; …when obtaining a video of a user shooting a pet, the sound collected during shooting can also be obtained, and the sound corresponding to each video frame in the action picture set corresponding to each expression is merged and saved in the order of the video frame to obtain each action picture Set the corresponding sound data…)
a model construction step, in which a data processing module (Par. 0091-100; Recognize the type of pet in the pet picture or video through the pre-trained pet type recognition model, and call the corresponding standard 3D model according to the recognized type…acquiring multiple pet picture samples for each pet to form a training sample set; for each pet picture sample in the training sample set Annotation processing is performed to annotate the type of pet in each pet image sample; the neural network is trained through the training sample set after the annotation processing to obtain a pet type recognition model. The pet picture taken by the user or the video frame in the video is input into the pet type recognition model, and the pet type recognition model can output the pet type after the recognition processing…Acquire biometric data of the pet from the pet picture or video, and combine the biometric data with the standard 3D model to generate a 3D model of the pet…), specific behavior features (Par. 0107-0110; after performing facial expression recognition on the pet picture or video through the pre-established facial expression recognition model in step S4, multiple continuously shot pictures or video frames recognized as the same facial expression may also be combined into the action corresponding to the facial expression Picture collection. For example, if the user takes 10 pet pictures in a row, the pictures 1-5 are recognized as expression A, and pictures 6-10 are recognized as expression B, then the pictures 1-5 are combined into the action corresponding to expression A Picture set 1, combine the 6th-10th pictures into the action picture set 2 corresponding to the expression B…the action picture set corresponding to the expression set by the user can be obtained, and multiple pictures in the obtained action picture set can be synthesized into the 3D model of the pet to generate the pet A 3D action image with the expression, and specific voice features of the specific animal (Par. 0111-0114; …when obtaining a video of a user shooting a pet, the sound collected during shooting can also be obtained, and the sound corresponding to each video frame in the action picture set corresponding to each expression is merged and saved in the order of the video frame to obtain each action picture Set the corresponding sound data. For example, if the user takes a 10-second video, the video frame from 1-5 seconds is recognized as expression A, and the video frame from 6-10 seconds is recognized as expression B, and the video frames from 1-5 seconds are recognized as expression B. The frame is combined into action picture set 1 corresponding to expression A, and the collected sounds of 1-5 seconds are saved as sound data corresponding to action set 1, and the video frames of 6-10 seconds are combined into the action picture corresponding to expression B Set 2, at the same time, save the collected sounds of the 6th-10th second as the sound data corresponding to action set 2…) to generate a specific 3D appearance model, a specific behavior model, and a specific voice model for the specific animal (Fig. 1-2 and Par. 0096; combine the biometric data with the standard 3D model to generate a 3D model of the pet…Par. 0101; In the 3D model of the pet, a 3D image of the pet with the expression is generated…Par. 0109; the obtained action picture set can be synthesized into the 3D model of the pet to generate the pet A 3D action image with the expression…Par. 0111; when generating a 3D action image of the pet with the expression, corresponding sounds, such as a call, can be used to generate the 3D image with a voice of the pet with the expression…Par. 0113; after the user sets the pet expression, the action picture set corresponding to the expression set by the user and the corresponding sound data can be obtained, and the video frames in the obtained action picture set can be synthesized into the 3D model of the pet), and stores the generated models in the customization database (Par. 0129; The standard 3D model corresponding to the identified category is called from a pre-established standard 3D model database, wherein the standard 3D model database stores multiple standard 3D models corresponding to pet categories. Par. 0138; Calling the action 3D model corresponding to the pet category from a pre-established standard 3D model database, wherein the standard 3D model database stores multiple action 3D models corresponding to the pet category. Par. 0148; Call the action 3D model and corresponding sound data corresponding to the type of pet from a pre-established standard 3D model database, wherein the standard 3D model database stores action 3D models and corresponding sound data corresponding to multiple pet types);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Xu’s model and customization database into Lim’s server-based virtual pet system. The motivation lies in the advantage of centralizing and maintaining the information used to generate and update the virtual pet, allows efficient retrieval of a relevant base model, and enables reuse of existing appearance, action, and sound information data gathered rather than reprocessing the data gathered whenever the virtual pet is generated, accessed, or refined. Further, separately maintaining specified models and their associated data would further allow efficient organization across multiple virtual pets while preserving the distinguishing characteristics of each particular animal. This organization would have constituted a predictable use of databases for their established functions of storing, organizing, retrieving, reusing, and updating information, yielding predictable results in reduced repetitive processing and storage and improving the overall efficiency and consistency of model generation and modification in virtual avatar/animal generation.
Regarding claim 11, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 10, and Lim further discloses wherein the terminal device further comprises a video camera for capturing real-time image data from the real world (Fig. 2; camera 160. Par. 0040; The camera 160 may be connected to the processor 130 and capture an image of a front object. Fig. 5 and Par. 0057; the user's terminal can start creating a companion animal avatar by taking a photograph of an actual companion animal using a camera in the terminal;
a microphone for capturing real-time audio data from the real world (Par. 0140; the input device may include a microphone. The method may further include transmitting a voice message for the user's companion animal avatar acquired through the microphone to the server, receiving a reply message of the companion animal avatar generated based on the voice message and the personality information from the server, and outputting the reply message in video and audio form through the output device as the companion animal avatar speaks);
and the video camera may include a motion sensing module for capturing motion data of at least one object within the real-time image data and transmitting the motion data to the server (Par. 0015; The terminal includes a transceiver, a memory, a processor, an input device, an output device, and a camera. The method includes a process of transmitting an image of a companion animal acquired using the camera to a server, a process of receiving first motion information about a motion of a companion animal avatar generated based on the image from the server in a virtual space, a process of outputting a motion of the companion animal avatar in the virtual space through the output device based on the first motion information…).
Lim and Xu are combined for the reasons set forth above with respect to claim 10.
Regarding claim 12, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 11, and Lim further discloses wherein the terminal device further comprises a video camera for capturing real-time image data from the real world (Fig. 2; camera 160. Par. 0040; The camera 160 may be connected to the processor 130 and capture an image of a front object. Fig. 5 and Par. 0057; the user's terminal can start creating a companion animal avatar by taking a photograph of an actual companion animal using a camera in the terminal;
a microphone for capturing real-time audio data from the real world (Par. 0140; the input device may include a microphone. The method may further include transmitting a voice message for the user's companion animal avatar acquired through the microphone to the server, receiving a reply message of the companion animal avatar generated based on the voice message and the personality information from the server, and outputting the reply message in video and audio form through the output device as the companion animal avatar speaks);
and the video camera may include a motion sensing module for capturing motion data of at least one object within the real-time image data and transmitting the motion data to the server (Par. 0015; The terminal includes a transceiver, a memory, a processor, an input device, an output device, and a camera. The method includes a process of transmitting an image of a companion animal acquired using the camera to a server, a process of receiving first motion information about a motion of a companion animal avatar generated based on the image from the server in a virtual space, a process of outputting a motion of the companion animal avatar in the virtual space through the output device based on the first motion information…).
Lim and Xu are combined for the reasons set forth above with respect to claim 10.
Regarding claim 13, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 12, but Lim does not disclose wherein the model optimization step uses a generative adversarial network (GAN).
Xu discloses wherein the model optimization step uses a generative adversarial network (GAN) (Par. 0099-0100; the 3D-GAN model can be used to adjust the size of the pet in the standard 3D model according to the mapping of the two-dimensional key point coordinates on the user's pet picture to the coordinates of the three-dimensional space…The foregoing synthesis of facial features, skin and hair features, limb features, and/or tail features of the pet on the pet in the standard 3D model can be specifically implemented through image synthesis software or models such as Deepfake…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to use Xu’s generative adversarial network into the model of Lim’s virtual pet system. The motivation lies in the advantage of automatically refining the standard model, improving the correspondence between the generated 3D model and the real pet, reducing the need for manual model adjustment, and producing a more accurate and realistic virtual identity using known generative modeling and deep learning techniques.
Regarding claim 14, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 12, and Lim further discloses wherein each 3D appearance model and specific 3D appearance model respectively include at least one or a combination of the following parameters: breed (Par. 0059; The terminal may identify the detailed species of the companion animal based on the determined type of companion animal…), gender, age, skin color, fur color, markings, body size, and physical structure (Par. 0061; The terminal may perform detailed modifications to the companion animal avatar by additionally reflecting detailed appearance characteristics. For example, information about hair color, eye size and distance, nose size, etc.);
each behavior model and specific behavior model respectively include at least one or a combination of the following parameters: sleeping, foraging, communicating, entertaining, learning, attacking, and defending (Par. 0069; the companion animal avatar can perform one or more of the following actions: 1) eat everything even when full, 2) serve food autonomously, and 3) eat as much as you can eat and keep others from eating. Par. 0080; given a situation where you meet a dog that is bigger than your companion animal avatar on a walk, the companion animal avatar can perform one or more of the following actions: 1) turn a blind eye, 2) bark aggressively, and 3) wonder and greet you and greet your scent. Par. 0088-0090; For example, in personality traits, an aggressive dog that barks furiously when meeting a new dog will behave that way if it happens to encounter another dog within the metaverse. For example, in personality traits, when the owner goes out, that is, when the user does not access the companion animal application in the terminal, the dog that waits only at the door waits with the corresponding behavior pattern while the owner does not access the app. For example, in behavioral characteristics, a dog that continues to eat snacks even when full will get sick from overeating item snacks);
and each voice model and specific voice model respectively include at least one or a combination of the following parameters: pitch, frequency, volume, rhythm, continuity, and voice type (Par. 0109; I can complete a set that configures the chatbot scenario to suit my pet. In addition, the voice of the companion animal can be voiced with the voice tone selected by the user).
Lim and Xu are combined for the reasons set forth above with respect to claim 10.
Regarding claim 15, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 14, and Lim further discloses wherein the interaction step further includes a real-time analysis step, in which the real-time analysis module adjusts the parameters in the specific 3D appearance model, specific behavior model, and specific voice model based on real-time image data transmitted from the video camera, real-time audio data transmitted from the microphone, and/or motion data transmitted from the motion sensing module to the server (Par. 0135-0140; The terminal includes a transceiver, a memory, a processor, an input device, an output device, and a camera. The method includes a process of transmitting an image of a companion animal obtained using the camera to a server…the process of transmitting the feedback information of the first input or the second input for the first motion information to the server, the process of receiving the second motion information of the companion animal avatar generated based on the feedback information, and the process of outputting the motion of the companion animal avatar in the virtual space through the output device based on the second motion information…The method may further include transmitting a voice message for the user's companion animal avatar acquired through the microphone to the server, receiving a reply message of the companion animal avatar generated based on the voice message and the personality information from the server)..
Lim and Xu are combined for the reasons set forth above with respect to claim 10.
Regarding claim 16, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 15, and Lim further discloses wherein the real-time analysis step further comprises:
a step of perceiving user behavior, in which the video camera, the microphone, and/or the motion sensing module detect a movement, a gesture, and/or a voice and transmit the detected data to the server (Fig. 8 and Par. 0093-0114; FIG. 8 shows a process for learning so that a companion animal avatar in virtual reality can communicate with a user by voice, assuming what emotion and how to communicate with a person based on the behavior pattern and personality…);
a step of collecting user data, in which the real-time analysis module collects real-time user image data, real-time user audio data, and/or user motion data corresponding to the detected movement, gesture, and/or voice (Par. 0103; when a message spoken by a user to a companion animal avatar by voice through a microphone in the terminal is converted into text and transmitted to the server using the STT (speech to text) function of the processor in the terminal);
a step of analyzing user data, in which the real-time analysis module performs an emotional analysis on the collected real-time user image data, real- time user audio data, and/or user motion data (Par. 0093-0094; enable users to feel mutual sympathy with companion animal avatars in virtual reality by operating as a chatbot (chatter robot) based on the formed behavior patterns and personalities…Based on the chatbot scenario described above, when the user speaks by voice, the companion animal avatar can respond by voice…the terminal recognizes the user's voice as STT (speech to text) and transmits it to the server, so that the chatbot of the companion animal avatar can recognize the user's voice message. Par. 0122-0131; Figure 10 shows an example of a process for automatically matching users in the metaverse and providing customized content based on the personality and behavioral characteristics of companion animal avatars…the interaction of the companion animal avatar and the user may be matched. For example, a group function that matches the owner of each companion animal avatar with the creator at the companion animal training center in the metaverse virtual reality can be performed according to the personality type of the companion animal avatar. In other words, in the metaverse virtual reality, each user performs a mutual action through the companion animal avatars of each user, and an event that can be matched can be performed…);
a step of analyzing user data, in which the real-time analysis module performs an emotional analysis on the collected real-time user image data, real- time user audio data, and/or user motion data (Par. 0093-0094; enable users to feel mutual sympathy with companion animal avatars in virtual reality by operating as a chatbot (chatter robot) based on the formed behavior patterns and personalities…Based on the chatbot scenario described above, when the user speaks by voice, the companion animal avatar can respond by voice…the terminal recognizes the user's voice as STT (speech to text) and transmits it to the server, so that the chatbot of the companion animal avatar can recognize the user's voice message. Par. 0122-0131; Figure 10 shows an example of a process for automatically matching users in the metaverse and providing customized content based on the personality and behavioral characteristics of companion animal avatars…the interaction of the companion animal avatar and the user may be matched. For example, a group function that matches the owner of each companion animal avatar with the creator at the companion animal training center in the metaverse virtual reality can be performed according to the personality type of the companion animal avatar. In other words, in the metaverse virtual reality, each user performs a mutual action through the companion animal avatars of each user, and an event that can be matched can be performed…);
a step of generating user command, in which the real-time analysis module generates a corresponding user command for the virtual identity based on the emotional analysis results (Par. 0103; when a message spoken by a user to a companion animal avatar by voice through a microphone in the terminal is converted into text and transmitted to the server using the STT (speech to text) function of the processor in the terminal, the server generates a response message based on the previously learned dialogue set and transmits it to the terminal, and the terminal converts the corresponding response message to speech as the companion animal avatar actually speaks through an output device and outputs moving screen and sound while outputting it); and
a step of adjusting behavior and/or adjusting emotional expression, in which the virtual identity adjusts various parameters in the specific 3D appearance model, specific behavior model, and specific voice model after receiving the user command based on the emotional analysis results (Par. 0102-0114; Based on the completed conversation set, the server can learn in which direction the conversation will proceed when the companion animal avatar has a conversation later…The companion animal avatar's answer may be selected according to the chatbot's expected answer of the companion animal selected by the user through positive (buy-in) or negative (buy-out) feedback. The server can learn based on the expected answer selected by the user…Answers of companion animal avatars generated by the server may be output as voices in text to speech (TTS) from the terminal. Through this, users can have the experience of having a conversation with a companion animal avatar in the metaverse virtual reality).
Lim and Xu are combined for the reasons set forth above with respect to claim 10.
Regarding claim 17, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 15, and Lim further discloses wherein the real-time analysis step further comprises:
a step of collecting environmental data, in which, in which the real-time analysis module collects environmental change data from the virtual environment (Par. 0051; The companion animal avatar characteristic simulation set formed through mutual information exchange between the user 100 and the server 200 creates an avatar through accumulated data in the server 200 and secures a variety of simulations and options);
a step of analyzing environmental data, in which the real-time analysis module performs an environmental analysis on the collected environmental change data (Par. 0135; a process of receiving first motion information about a motion of a companion animal avatar generated based on the image from the server in a virtual space, a process of outputting a motion of the companion animal avatar in the virtual space through the output device based on the first motion information, and a first input indicating that the motion based on the first motion information of the companion animal avatar is similar to the actual motion of the companion animal through the input device, or indicating that it is not similar. The process of receiving feedback information of the second input, the process of transmitting the feedback information of the first input or the second input for the first motion information to the server, the process of receiving the second motion information of the companion animal avatar generated based on the feedback information, and the process of outputting the motion of the companion animal avatar in the virtual space through the output device based on the second motion information);
a step of generating an environmental command, in which the real-time analysis module generates a corresponding environmental command for the virtual identity based on the results of the environmental analysis (Par. 0072; The user may go beyond inputting feedback information and more actively create a simulation of a situation in which the companion animal avatar will operate. In a specific situation set by the user…Par. 0080; companion animal avatar on a walk…Par. 0106-0107; a conversation system can be established in a space where companion animals can communicate…For example, in a metaverse space implemented like a counseling center or cafe, users can try to talk with companion animals. Par. 0125; events in which companion animal avatars whose personality types are determined by the server interact in the metaverse virtual space may be generated); and
a step of adjusting behavior and/or expressing emotions, in which the virtual identity adjusts various parameters in the specific 3D appearance model, specific behavior model, and specific voice model after receiving the environmental command based on the results of the environmental analysis (Par. 0062-0063; Generation of the companion animal avatar can be completed by receiving a confirmation input based on a comparison of the appearance of the companion animal created by the user and the actual companion animal at each stage of the companion animal avatar creation process…created companion animal avatar can operate in the virtual world in a 3D form. Par. 0070-0074; User can input positive or negative feedback about this. Feedback input by the user may be the basis for determining how the server will behave in a similar situation with respect to the companion animal avatar in the future…In a specific situation set by the user, by seeing how the current companion animal behaves based on the preset personality information, the user can provide additional feedback and gradually learn the companion animal avatar to be similar to the actual companion animal…Feedback input from the user may be reflected to the companion animal avatar by the server after the simulation is finished…Par. 0102-0114; Based on the completed conversation set, the server can learn in which direction the conversation will proceed when the companion animal avatar has a conversation later…The companion animal avatar's answer may be selected according to the chatbot's expected answer of the companion animal selected by the user through positive (buy-in) or negative (buy-out) feedback. The server can learn based on the expected answer selected by the user…Answers of companion animal avatars generated by the server may be output as voices in text to speech (TTS) from the terminal. Through this, users can have the experience of having a conversation with a companion animal avatar in the metaverse virtual reality).
Lim and Xu are combined for the reasons set forth above with respect to claim 10.
Regarding claim 18, Lim in view of Xu discloses the method for implementing a pet reconstruction system based on a virtual world infrastructure as claimed in claim 15, and Lim further discloses wherein the real-time analysis step further comprises:
a step of collecting virtual animal data, in which the real-time analysis module collects behavioral and emotional data of at least one other virtual identity within the virtual environment (Par. 0123; When a companion animal avatar created in the metaverse virtual reality comes into contact with other companion animal avatars, the preset behavior and personality characteristics of each avatar are internalized and interacted with.);
a step of analyzing virtual animal data, in which the real-time analysis module performs a virtual animal analysis on the collected behavioral and emotional data of the other virtual identity (Par. 0080; given a situation where you meet a dog that is bigger than your companion animal avatar on a walk, the companion animal avatar can perform one or more of the following actions: 1) turn a blind eye, 2) bark aggressively, and 3) wonder and greet you and greet your scent. Par. 0088-0090; For example, in personality traits, an aggressive dog that barks furiously when meeting a new dog will behave that way if it happens to encounter another dog within the metaverse. For example, in personality traits, when the owner goes out, that is, when the user does not access the companion animal application in the terminal, the dog that waits only at the door waits with the corresponding behavior pattern while the owner does not access the app. Par. 0124-0125; depending on the personality type of the companion animal, the interaction between avatars of the companion animal may be implemented as an artificial intelligence event…The personality type of a companion animal is one of a set number of personality types, such as the Myers-Briggs-Type Indicator (MBTI), which classifies a person's personality into 16 limited types, and each companion animal avatar in the metaverse virtual reality can be classified according to behavior and personality characteristics. Par. 0125-0128; Thereafter, events in which companion animal avatars whose personality types are determined by the server interact in the metaverse virtual space may be generated…);
a step of generating a virtual animal command, in which the real-time analysis module generates a corresponding virtual animal command for the virtual identity based on the results of the virtual animal analysis (Par. 0081-0083; User can input positive or negative feedback about this. Feedback input by the user may be the basis for determining how the server will behave in a similar situation with respect to the companion animal avatar in the future…); and
a step of adjusting behavior and/or a step of adjusting emotional expression, in which the virtual identity modifies various parameters in the specific 3D appearance model, specific behavior model, and specific voice model after receiving the virtual animal command based on the results of the virtual animal analysis (Par. 0083; The user may more actively create a simulation of a situation in which the companion animal avatar will operate beyond inputting feedback information. In a specific situation set by the user, by seeing how the current companion animal behaves based on the preset personality information, the user can provide additional feedback and gradually learn the companion animal avatar to resemble the actual companion animal. Par. 0126-0130).
Lim and Xu are combined for the reasons set forth above with respect to claim 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JENNY NGAN TRAN whose telephone number is (571)272-6888. The examiner can normally be reached Mon-Thurs 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JENNY N TRAN/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights