Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 30, 2026 has been entered.
Claim Rejections - 35 USC § 112
Claims 1-6 and 8-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. In line 9 of claim 1, “an image” is indefinite because it is unclear as to whether “an image” in line 7 is referenced or an entirely different image. Similar indefiniteness exists in claims 11 and 15.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5 and 8-15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kim et al (US 2017/0212585).
As per claim 1 Kim et al depicts in figure 1 and discloses, insofar as the claims are definite and understood: An electronic device 100 comprising:
a display 160; at least one camera 180; a memory 130 storing at least one instruction; and at least one processor 120 / 301 configured to execute the at least one instruction to cause the electronic device 100 to {figure 1}:
obtain spatial information about a real-world space based on an image obtained through the at least one camera 180 {figure 3B: 301};
obtain user inputs based on the image obtained through the at least one camera 180;
obtain a two-dimensional (2D) guide image 731 {figure 7 & [0107] The processor 120 can recognize a user input of the user using the camera module 180 or the sensor module 190. The user input can be represented as a 2D coordinate or a 3D coordinate.};
obtain object characteristic information 741 including at least one of a type, a shape, a color, a location, a size, or a theme of an object from the user inputs, and the 2D guide image 731 {figure 7 & [0052] For example, the data about the real object can include location information of each of real objects included in an image collected by the camera module 180 and distance information and relative distance information from the camera module 180, and the like. The data about the virtual object can include a size and a location of each of virtual objects output together with real objects generated by an input of the user, information about correlation with the real objects, and the like.};
obtain object generation information for generating a virtual object, based on the spatial information and the object characteristic information {figure 3B: 302},
generate the virtual object for the object generation information by inputting the object generation information to a generative artificial intelligence (AI) model trained to generate a three-dimensional (3D) virtual object based on information about a space and an object {figure 3B: 304 & [0087] The virtual object modeling unit 301e can model a virtual model operated by the operation object used by the user to generate a 3D virtual rendering space including the modeled virtual object. Location information of the virtual object modeled by the virtual object modeling unit 301e can be represented as a 3D coordinate on the 3D rendering space. Also, the virtual object modeling unit 301e can add physical characteristic information in a weightless state of a virtual object to model the virtual object.}, and
control the display 160 to display 160 the virtual object {figure 3B: 305}.
As per claim 2 Kim et al discloses, insofar as the claims are definite and understood: The electronic device 100 of claim 1, wherein the at least one camera 180 comprises a first camera 180 configured to obtain a spatial image of the real-world space by capturing an image of the real-world space, and wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to:
obtain the spatial information regarding at least one of a type, a category, a color, a theme, or an atmosphere of the real-world space from the spatial image obtained through the first camera 180 {[0069] The camera module 180 can collect image data in real time from an AR output. The camera module 180 can include at least one red, green, and blue (RGB) cameras, an infrared (IR) camera, a depth camera, or a thermal camera. In various embodiments, the image data can be generated by combining data collected by a plurality of cameras.}.
As per claim 3 Kim et al discloses, insofar as the claims are definite and understood: The electronic device 100 of claim 1, wherein the at least one camera 180 comprises a second camera 180 configured to obtain a hand image by capturing an image of a hand of a user, and wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to:
recognize a gesture input from the user in the hand image obtained through the second camera 180, and extract the object characteristic information 741 regarding at least one of the shape, the location, or the size of the object from the gesture input { figure 7 & [0077] In various embodiments, an image collected by the camera module 211 can be used to generate a depth map and recognize a gesture of the user. Also, the camera module 211 can be used for 3D modeling/mapping of an object around the electronic device 201.}.
As per claim 4 Kim et al discloses, insofar as the claims are definite and understood: The electronic device 100 of claim 3, wherein the second camera 211 is configured as a depth camera 211 comprising at least one of a time-of-flight (ToF) camera, a stereo vision camera 211, or a light detection and ranging (LiDAR) sensor, and is configured to obtain a depth image by capturing the image of the hand of the user {[0076] In an embodiment, the camera module 211 can be a stereo camera. The stereo camera can capture one object to simultaneously obtain two images by mounting two capturing lenses. In another embodiment, the camera module 211 can include an IR output device and an IR camera. The IR output device can emit infrared rays to an object around the electronic device 201, and the IR camera can sense infrared rays reflected from the object.}, and wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to:
recognize the gesture input from the user in the depth image obtained through the second camera 211 {[0077] In various embodiments, an image collected by the camera module 211 can be used to generate a depth map and recognize a gesture of the user.}.
As per claim 5 Kim et al discloses, insofar as the claims are definite and understood: The electronic device 100 of claim 1, further comprising: a touch screen configured to receive a touch input from a user, wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to:
recognize a gesture input from the touch input received through the touch screen { [0059] The display 160 can include a touch screen, and can receive, for example, a touch, a gesture, proximity, or a hovering input using an electronic pen or part of a body of the user.}, and
extract the object characteristic information 741 regarding at least one of the shape, the location, or the size of the object from the gesture input { figure 7 & [0103] For example, the user input can be menu selection on a graphic user interface (GUI). For another example, the user input can be an input according to a specific gesture or a specific input pattern of the user.}.
As per claim 8 Kim et al discloses, insofar as the claims are definite and understood: The electronic device 100 of claim 1, wherein the object characteristic information comprises at least one of first object characteristic information obtained from spatial information, second object characteristic information obtained from a gesture input, third characteristic information obtained from a voice input, or fourth characteristic information obtained from the 2D guide image {Note: the fourth characteristic set forth by way of alternative language is disclosed}, wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to:
convert the first object characteristic information into first feature data by performing vector embedding on the spatial information, convert the second object characteristic information into second feature data by performing vector embedding on the second object characteristic information obtained from the gesture input, convert the third object characteristic information into third feature data by performing vector embedding on the third object characteristic information obtained from the voice input, convert the fourth object characteristic information into fourth feature data by performing vector embedding on the fourth object characteristic information obtained from the 2D guide image, and obtain feature data representing the object generation information based on the first to fourth feature data { [0190] In an embodiment, the processor 120 can extract a second portion in the 3D space information included in a range corresponding to a path of a user input and can measure a distance between the second portion and the path of the user input. For example, if a plane normal vector corresponding to the path of the user input is x and if a plane normal vector corresponding to the second portion is y, a distance between the path of the user input and the second portion can be calculated through a vector inner product (x.Math.y).}.
As per claim 9 Kim et al discloses, insofar as the claims are definite and understood: The electronic device 100 of claim 8, wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to: modify the object generation information based on the user inputs {figure 3C: 350 & figure 7}.
As per claim 10 Kim et al discloses since the alternative language is set forth, insofar as the claims are definite and understood: The electronic device 100 of claim 9, wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to: modify the object generation information by adjusting, based on the user inputs, a weight value assigned to each of the first object characteristic information, the second object characteristic information extracted from the gesture input, the third object characteristic information obtained from the voice input, and the fourth object characteristic information obtained from the 2D guide image {figure 3C: 350 & figure 7}.
As per claim 11 Kim et al discloses, insofar as the claims are definite and understood: A method, performed by an electronic device 100, of generating a virtual object, the method comprising:
obtaining user inputs based on an image obtained through at least one camera 180 {figure 1};
obtaining spatial information about a real-world space based on the image obtained through the at least one camera 180 {figure 3B: 301};
obtain a two-dimensional (2D) guide image 731 {figure 7 & [0107] The processor 120 can recognize a user input of the user using the camera module 180 or the sensor module 190. The user input can be represented as a 2D coordinate or a 3D coordinate.};
obtaining object characteristic information 741 including at least one of a type, a shape, a color, a location, a size, or a theme of an object from the user inputs and 2D guide image 731 { figure 7 & [0052] For example, the data about the real object can include location information of each of real objects included in an image collected by the camera module 180 and distance information and relative distance information from the camera module 180, and the like. The data about the virtual object can include a size and a location of each of virtual objects output together with real objects generated by an input of the user, information about correlation with the real objects, and the like.};
obtaining object generation information for generating the virtual object, based on the spatial information and the object characteristic information {figure 3B: 302};
generating the virtual object for the object generation information by inputting the object generation information to a generative artificial intelligence (AI) model trained to generate a three-dimensional (3D) virtual object based on information about a space and an object {figure 3B: 304 & [0087] The virtual object modeling unit 301e can model a virtual model operated by the operation object used by the user to generate a 3D virtual rendering space including the modeled virtual object. Location information of the virtual object modeled by the virtual object modeling unit 301e can be represented as a 3D coordinate on the 3D rendering space. Also, the virtual object modeling unit 301e can add physical characteristic information in a weightless state of a virtual object to model the virtual object.}; and
displaying, with a display 160, the virtual object {figure 3B: 305}.
As per claim 12 Kim et al discloses, insofar as the claims are definite and understood: The method of claim 11, wherein the object characteristic information comprises at least one of first object characteristic information obtained from spatial information, second object characteristic information obtained from a gesture input, third characteristic information obtained from a voice input, or fourth characteristic information obtained from the 2D guide image {Note: the fourth characteristic set forth by way of alternative language is disclosed}, and wherein the generating of the object generation information comprises:
converting the first object characteristic information into first feature data by performing vector embedding on the spatial information; converting the second object characteristic information into second feature data by performing vector embedding on the second object characteristic information obtained from the gesture input; converting the third object characteristic information into third feature data by performing vector embedding on the third object characteristic information obtained from the voice input; converting the fourth object characteristic information into fourth feature data by performing vector embedding on the fourth object characteristic information obtained from the 2D guide image; and obtaining feature data representing the object generation information based on the first to fourth feature data { [0190] In an embodiment, the processor 120 can extract a second portion in the 3D space information included in a range corresponding to a path of a user input and can measure a distance between the second portion and the path of the user input. For example, if a plane normal vector corresponding to the path of the user input is x and if a plane normal vector corresponding to the second portion is y, a distance between the path of the user input and the second portion can be calculated through a vector inner product (x.Math.y).}.
As per claim 13 Kim et al discloses, insofar as the claims are definite and understood: The method of claim 12, further comprising: receiving the user inputs for modifying the object generation information; and modifying the object generation information based on the user inputs {figure 3C: 350 & figure 7}.
As per claim 14 Kim et al discloses, insofar as the claims are definite and understood: The method of claim 13, wherein the modifying of the object generation information comprises modifying the object generation information by adjusting, based on the user inputs, a weight value assigned to each of the first object characteristic information, the second object characteristic information obtained from the gesture input, the third object characteristic information obtained from the voice input, and the fourth object characteristic information obtained from the 2D guide image {figure 3C: 350 & figure 7}.
As per claim 15 Kim et al discloses, insofar as the claims are definite and understood: A computer program product comprising a computer-readable storage medium, wherein the computer-readable storage medium comprises instructions that are readable by an electronic device 100 to:
obtain user inputs based on an image obtained through at least one camera 180 {figure 1};
obtain spatial information about a real-world space based on the image obtained through at least one camera 180 {figure 3B: 301};
obtain a two-dimensional (2D) guide image 731 {figure 7 & [0107] The processor 120 can recognize a user input of the user using the camera module 180 or the sensor module 190. The user input can be represented as a 2D coordinate or a 3D coordinate.};
obtain object characteristic information 741 including at least one of a type, a shape, a color, a location, a size, or a theme of an object from the user inputs and the 2D guide image 731 { figure 7 & [0052] For example, the data about the real object can include location information of each of real objects included in an image collected by the camera module 180 and distance information and relative distance information from the camera module 180, and the like. The data about the virtual object can include a size and a location of each of virtual objects output together with real objects generated by an input of the user, information about correlation with the real objects, and the like.};
obtain object generation information for generating a virtual object, based on the spatial information and the object characteristic information {figure 3B: 302};
generate the virtual object for the object generation information by inputting the object generation information to a generative artificial intelligence (AI) model trained to generate a three-dimensional (3D) virtual object based on information about a space and an object {figure 3B: 304 & [0087] The virtual object modeling unit 301e can model a virtual model operated by the operation object used by the user to generate a 3D virtual rendering space including the modeled virtual object. Location information of the virtual object modeled by the virtual object modeling unit 301e can be represented as a 3D coordinate on the 3D rendering space. Also, the virtual object modeling unit 301e can add physical characteristic information in a weightless state of a virtual object to model the virtual object.}; and
display, with a display 160, the virtual object {figure 3B: 305}.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Kim et al (US 2017/0212585) in view of Fallon (US 2018/0004481).
As per claim 6 Kim et al discloses, insofar as the claims are definite and understood: The electronic device 100 of claim 1, further comprising: a microphone configured to receive a voice input from a user, wherein the at least one processor 120 / 301 is further configured to execute the at least one instruction to cause the electronic device 100 to:
obtain a speech signal from the voice input { [0246] The processor 120 can use ambient information such as a voice during an input operation.} received through the microphone { [0335] The audio module 1980 can interchangeably convert a sound into an electric signal. At least some of components of the audio module 1980 can be included in, for example, an I/O interface 150 shown in FIG. 1. The audio module 1980 can process sound information input or output through, for example, a speaker 1982, a receiver 1984, an earphone 1986, or the microphone 1988, and the like. }
Regarding claim 6 Kim et al is silent as to: converting the speech signal into text; and extracting the object characteristic information comprising at least one of a type, a shape, a color, or a theme of the object by analyzing the text using a natural language understanding (NLU) model. With respect to claim 6 Fallon depicts in figure 2: converting the speech signal into text {figure 2: 226}; and extracting the object characteristic information comprising at least one of a type, a shape, a color, or a theme of the object by analyzing the text using a natural language understanding (NLU) model {figure 2: 228}.
It would have been obvious to a person having ordinary skill in the art at the time the invention was effectively filed to have the electronic device of Kim et al convert the speech signal into text; and extract the object characteristic information comprising at least one of a type, a shape, a color, or a theme of the object by analyzing the text using a natural language understanding (NLU) model. The rationale is as follows: one of ordinary skill in the art at the time the invention was effectively filed would have been motivated to have an electronic device to convert the speech signal into text; and extract the object characteristic information comprising at least one of a type, a shape, a color, or a theme of the object by analyzing the text using a natural language understanding (NLU) model so as to provide a user friendly hands-free input by using voice commands.
Response to Arguments
Applicant's arguments filed April 30, 2026 have been fully considered but they are not persuasive. In the paragraph bridging pages 2 and 3 applicant asserts the following:
In other words, the cited reference Kim does not disclose or imply any technical features corresponding to i) "obtaining the two-dimensional (2D) guide image" and ii) obtaining object characteristic information including at least one of a type, a shape, a color, a location, a size, or a theme of an object from the user inputs and the 2D guide image" as recited in the amended claim 1 of the present application. In addition, the client would like to amend the other independent claim 11 and a computer program product claim 15 in the same manner as done for claim 1.
As stated supra, Kim et al does disclose “obtaining the two-dimensional (2D) guide image” in [0107], which states “The processor 120 can recognize a user input of the user using the camera module 180 or the sensor module 190. The user input can be represented as a 2D coordinate or a 3D coordinate.” Additionally, Kim et al discloses “obtaining object characteristic information including at least one of a type, a shape, a color, a location, a size, or a theme of an object from the user inputs and the 2D guide image" in figure 7 and in [0052], which states:
For example, the data about the real object can include location information of each of real objects included in an image collected by the camera module 180 and distance information and relative distance information from the camera module 180, and the like. The data about the virtual object can include a size and a location of each of virtual objects output together with real objects generated by an input of the user, information about correlation with the real objects, and the like.
Therefore, contrary to applicant’s assertion Kim et al discloses the claimed invention.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID D DAVIS whose telephone number is (571)272-7572. The examiner can normally be reached Monday - Friday, 8 a.m. - 4 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached at 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID D DAVIS/Primary Examiner, Art Unit 2627
DDD
Prosecution Insights