Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-13, 15, 16, 19, and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kurz (U.S. Patent No. 11,544,865), referred herein as Kurz.
Regarding claim 1, Kurtz teaches an image processing apparatus comprising one or more memories storing instructions and one or more processors that execute the instructions to (figs 2 or 3; column 2, line 63 through column 3, line 2):
obtain a three-dimensional model of an object (column 11, lines 24-35 and 56-67; column 13, lines 57-60; column 13, line 65 through column 14, line 2; a 3D model of a subject person and surrounding environment is captured);
obtain information indicating a situation of the object represented by the three-dimensional model (column 12, lines 12-18; column 13, line 65 through column 14, line 9; information is obtained indicating a situation of the subject person in the environment, such as information indicating a posture of the subject person);
determine whether the information indicating the situation of the object represented by the three-dimensional model satisfies a specific condition (column 12, lines 12-27; column 13, line 65 through column 14, line 9; the system determines whether the posture information satisfies a specific condition, such as whether the posture is good or bad); and
generate a virtual viewpoint image of the object, wherein the virtual viewpoint image includes an image of the object according to the three-dimensional model of the object (fig 5, virtual viewpoint depictions 504/516/520/522; column 13, lines 57-60; column 13, line 65 through column 14, line 4; column 14, lines 28-32; a virtual viewpoint image of the object is generated that includes an image of the object according to its 3D model),
wherein generating the virtual viewpoint image includes changing an appearance of the object in the virtual viewpoint image or an appearance of surroundings of the object in the virtual viewpoint image in response to a determination that the information indicating the situation of the object satisfies the specific condition (fig 5, virtual viewpoint depictions 504/516/520/522; column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 13, line 65 through column 14, line 4; column 14, lines 10-32; if the specific condition is satisfied, such as when the posture is determined to be bad, the appearance of the object may be changed, for example, to show a depiction of the object with correct posture, and/or and appearance of the surroundings of the object may be changed, for example, to insert messages into the surrounding environment, or insert obstacles or virtual objects into the surrounding environment).
Regarding claim 2, Kurtz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: estimate a posture of the object based on captured images of the object captured by each of a plurality of cameras (column 11, lines 24-35 and 39-66; column 13, lines 55-67).
Regarding claim 3, Kurtz teaches the image processing apparatus according to claim 2, wherein the object is a person, and the one or more processors execute the instructions to estimate the posture of the object based on a position of a joint of the person (column 11, lines 24-35; column 13, line 57 through column 14, line 20).
Regarding claim 4, Kurtz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: generate the information indicating the situation of the object based on a captured image of the object (column 11, lines 24-35; column 12, lines 12-18; column 13, lines 57-67; column 14, lines 5-20).
Regarding claim 5, Kurtz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: generate the information indicating the situation of the object based on a result of determining a posture of the object (column 12, lines 12-18; column 14, lines 5-20).
Regarding claim 6, Kurtz teaches the image processing apparatus according to claim 1, wherein the information indicating the situation of the object is information calculated based on a change in a position, a posture, a shape, or an appearance of the object across a plurality of times (column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60).
Regarding claim 7, Kurtz teaches the image processing apparatus according to claim 1, wherein the information indicating the situation of the object is information indicating an event that has occurred in relation to the object (column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60; an event such as movement and/or posture changes is indicated).
Regarding claim 8, Kurtz teaches the image processing apparatus according to claim 1, wherein the information indicating the situation of the object is information indicating movement of the object, a change in a posture of the object, a change in a shape of the object, or a result of classifying the situation (column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60; movement and/or postures changes are indicated, and a result of classifying the posture is indicated).
Regarding claim 9, Kurtz teaches the image processing apparatus according to claim 1, wherein the information indicating the situation of the object is information indicating that a specific movement or posture change has occurred in the object (column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60; movement and/or postures changes are indicated).
Regarding claim 10, Kurtz teaches the image processing apparatus according to claim 1, wherein the information indicating the situation of the object is information indicating a result of analyzing movement of the object (column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60; the indication is the result of analyzing and classifying movement and posture of the object).
Regarding claim 11, Kurtz teaches the image processing apparatus according to claim 1, wherein the information indicating the situation of the object is information indicating a movement speed or an amount of deformation of a shape of the object (column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60; deformation of joint positions resulting from movement and/or posture changes is indicated).
Regarding claim 12, Kurtz teaches the image processing apparatus according to claim 1, wherein the information indicating the situation of the object is information specifying positions of the object in time series, or information indicating a posture of the object (column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60; posture and positions in time series are indicated).
Regarding claim 13, Kurtz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: change a display pertaining to the object in the virtual viewpoint image in accordance with the information indicating the situation of the object (column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues are added to the display to indicate the situation of the subject person).
Regarding claim 15, Kurtz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: add an object item in a periphery of the object in accordance with the information indicating the situation of the object (column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are added in the periphery of the subject person to indicate the situation).
Regarding claim 16, Kurtz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: superimpose the information indicating the situation of the object on the virtual viewpoint image (column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are superimposed on the virtual viewpoint image to indicate the situation).
Regarding claim 19, the limitations of this claim substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Regarding claim 20, the limitations of this claim substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kurz, in view of Zhang et al. (U.S. Patent Application Publication No. 2017/0064214), referred herein as Zhang.
Regarding claim 14, Kurz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: change visual cues for the object in the virtual viewpoint image in accordance with the information indicating the situation of the object (column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are added to the virtual viewpoint image to indicate the situation).
Kurtz does not teach changing a size or a color of the object.
However, in a similar field of endeavor, Zhang teaches an apparatus configured to obtain a three-dimensional model of an object, obtain information indicating a situation of the object regarding changes in its posture and/or position, and generating a virtual viewpoint image according to the 3D model and the information indicating the situation (figs 3B and 20; paragraph 126; paragraphs 129 and 130; paragraph 134; paragraphs 136-138), wherein the apparatus is further configured to change a size or a color of the object in the virtual viewpoint image in accordance with the situation of the object (paragraph 307, lines 1-15).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the color or size change taught by Zhang with the visual cues of Kurz because this helps the subject person have a better understanding of how the posture needs to be adjusted, thereby increasing the usability and accuracy of the posture analysis procedures (see, for example, Zhang, paragraph 307, the last 10 lines).
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kurz, in view of Shear et al. (U.S. Patent Application Publication No. 2023/0376581), referred herein as Shear.
Regarding claim 17, Kurz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: determine whether a user corresponds to a display mode, and when the user is determined to correspond to the display mode, generate a virtual viewpoint image of the object, according to the information indicating the situation of the object, that corresponds to the display mode (column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are added to the virtual viewpoint image to indicate the situation, and can comprise various display modes such as virtual object addition, message indication, information superimposition, etc.).
Kurz does not teach determining whether a user has a right to the display mode.
However, in a similar field of endeavor, Shear teaches an apparatus configured to obtain a three-dimensional model of an object, and obtain information indicating a situation of the object regarding changes in its posture and/or position (paragraph 427, lines 1-9 and the last 11 lines; paragraph 433, lines 1-17; paragraph 459), wherein the apparatus is further configured to determine whether a user has a right to a display mode (paragraph 276, lines 1-14 and the last 18 lines; paragraph 282, the last 19 lines; paragraph 292).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the rights determination of Shear with the subject person analysis and display of Kurz because this facilitates the vetting and authorization of access to the subject person’s analyzed information, thereby providing a much higher degree of computing resource trustworthiness and increasing the accuracy, reliability, and security of the user’s situation evaluation (see, for example, Shear, paragraph 163; paragraph 178, lines 1-15; paragraph 433, lines 1-12).
Regarding claim 18, Kurz teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: determine whether a user corresponds to the information indicating the situation of the object, and when the user is determined to correspond to the information indicating the situation of the object, generate a virtual viewpoint image of the object according to the information indicating the situation of the object (column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32).
Kurz does not teach determining whether a user owns a non-fungible token (NFT) corresponding to the information.
However, in a similar field of endeavor, Shear teaches an apparatus configured to obtain a three-dimensional model of an object, and obtain information indicating a situation of the object regarding changes in its posture and/or position (paragraph 427, lines 1-9 and the last 11 lines; paragraph 433, lines 1-17; paragraph 459), wherein the apparatus is further configured to determine whether a user owns a non-fungible token (NFT) corresponding to the information (paragraph 162, lines 1-15; paragraph 1591).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the rights determination of Shear with the subject person analysis and display of Kurz because this facilitates the vetting and authorization of access to the subject person’s analyzed information, thereby providing a much higher degree of computing resource trustworthiness and increasing the accuracy, reliability, and security of the user’s situation evaluation (see, for example, Shear, paragraph 163; paragraph 178, lines 1-15; paragraph 433, lines 1-12).
Response to Arguments
Applicant’s arguments on page 7, with respect to the claim objections, have been fully considered and are persuasive. The amendments have overcome the claim objections; thus they are withdrawn.
Applicant’s arguments with respect to the prior art rejections have been fully considered, but they are not persuasive.
On pages 8 and 9 of the Applicant’s Remarks, with respect to the 102 rejection of claim 1, the Applicant argues, in summary, that 1) Kurz discloses determining a user’s posture based on an image of the user and provides feedback encouraging the user to change posture, but does not disclose determining whether information indicating the situation of an object represented by a 3D model satisfies a specific condition, and 2) Kurz does not teach the claimed generating step because Kurz is directed to posture analysis and corrective feedback, not to evaluating the situation of an object represented by a three-dimensional model and then changing the appearance of the object itself, or its surroundings in a virtual viewpoint image; and the displayed content of Kurz is fundamentally different from changing an appearance of the object or the appearance of surrounding of that object based on the condition. The Examiner respectfully disagrees with these arguments.
Regarding the first argument, it is respectfully submitted that determining “whether the information indicating the situation of the object…satisfies a specific condition” is a broad limitation with a number of reasonable interpretations. As just one example of a reasonable interpretation of this limitation in the provided citations to Kurz, the object (e.g. the user / subject person) is disclosed as being represented by a 3D model, information indicating a situation of the object (e.g. the user’s posture) is obtained, and the system determines whether the information indicating the situation satisfies a specific condition (e.g. determines whether the “posture condition” is good or bad). This “specific condition” is expressly recited in Kurz, and its classification as good or bad is just one reasonable interpretation of determining whether the “information indicating the situation…satisfies a specific condition.” Thus the Examiner respectfully submits that Kurz clearly teaches this limitation.
Regarding the second argument, it is also respectfully submitted that “changing an appearance of the object…or an appearance of surroundings of the object” is also quite broad with a number of reasonable interpretations, several of which are taught by Kurz. As noted above, Kurz expressly teaches determining whether the information indicating the situation satisfies a specific condition. In the provided citations, Kurz also teaches generating a virtual viewpoint image (e.g. images 504, 516, 520, or 522), and when the information indicating the situation satisfies the specific condition (e.g. when it is determined that the posture is bad), the appearance of the object in the virtual viewpoint image may be changed (e.g. the change shown in the object when going from image 504 to 516, or from 520 to 522, where the change depicts the user going from bad posture to a desired corrected posture), or the surroundings of the object in the virtual viewpoint image may be changed (e.g. by displaying a message, inserting an obstacle into the surroundings, or inserting a virtual object into the surroundings). Thus the Examiner respectfully submits that Kurz clearly teaches this limitation.
Accordingly, it is respectfully submitted that any potential differences between the claimed “situation,” “condition,” or “changing an appearance” and those disclosed by the prior art are not currently reflected in the claims. Thus, as claimed, Kurz teaches each and every limitation of amended claim 1, and the rejection is respectfully maintained.
On pages 9 and 10 of the Applicant's Remarks, the Applicant argues that the remaining independent claims are not taught by the prior art for reasons similar to those discussed in regard to claim 1, and that the dependent claims are not taught, insomuch as they depend from claims that are not taught by the prior art. The Examiner respectfully disagrees with these arguments, for the reasons discussed above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights