Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 11, 15, 17-22 are rejected under 35 U.S.C. 103 as being unpatentable over Kurz (U.S. Patent No. 11,544,865), referred herein as Kurz, in view of Shear et al. (U.S. Patent Application Publication No. 2023/0376581), referred herein as Shear.
Regarding claim 1, Kurtz teaches an image processing apparatus comprising one or more memories storing instructions and one or more processors that execute the instructions to (figs 2 or 3; column 2, line 63 through column 3, line 2):
obtain three-dimensional models of a specific object in time series (column 11, lines 24-35 and 56-67; column 12, lines 12-28; column 13, lines 57-60; column 13, line 65 through column 14, line 2; a 3D model of a subject person and surrounding environment in time series is captured);
obtain posture information of the specific object in time series (column 12, lines 12-28 and 58-67; column 13, line 65 through column 14, line 9; column 14, lines 5-20 and 47-60; posture information is obtained in time series);
detect a specific posture movement of the specific object represented by the three-dimensional model based on the posture information (column 12, lines 12-28 and 58-67; column 13, line 65 through column 14, line 9; column 14, lines 5-20 and 47-60; specific posture movement of the object represented by the 3D model is detected); and
generate a virtual viewpoint image of the specific object using the three-dimensional model of the specific object, wherein the virtual viewpoint image comprises a plurality of virtual viewpoint images respectively including images of the specific object rendered from the three-dimensional model of the specific object (fig 5, virtual viewpoint depictions 504/516/520/522; column 12, lines 12-28; column 13, lines 57-60; column 13, line 65 through column 14, line 4; column 14, lines 28-32; a series of virtual viewpoint images of the object are rendered from 3D model),
wherein generating the virtual viewpoint image includes changing an appearance of the specific object in virtual viewpoint images corresponding to a period during which the specific posture movement of the specific object is detected, in response to detecting the specific posture movement of the specific object (fig 5, virtual viewpoint depictions 504/516/520/522; column 12, lines 12-28 and 46-57; column 13, lines 3-10 and 21-32; column 13, line 65 through column 14, line 4; column 14, lines 5-32; the appearance of virtual viewpoint images are changed corresponding to the period in which the posture movement of the object is detected).
Kurtz teaches that the system may comprise images or video in time series (column 5, lines 41-50; column 6, lines 35-37; column 12, lines 12-28), but does not explicitly teach that the virtual viewpoint images of the object are a video.
However, in a similar field of endeavor, Shear teaches an apparatus configured to obtain a three-dimensional model of an object, and obtain information indicating a situation of the object regarding changes in its posture and/or position (paragraph 427, lines 1-9 and the last 11 lines; paragraph 433, lines 1-17; paragraph 459), wherein a virtual viewpoint video is generated for the object (paragraph 254, lines 1-16; paragraph 274, lines 1-10; paragraph 277, lines 14-32).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the object video of Shear with the time series images of Kurz because this helps ensure higher fidelity biometric evaluation and increased the accuracy, reliability, and security of the user’s situation evaluation (see, for example, Shear, paragraph 178, lines 1-15; paragraph 254, the last 7 lines; paragraph 277, the last 10 lines).
Regarding claim 2, Kurtz in view of Shear teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: estimate a posture of the specific object based on captured images of the specific object captured by each of a plurality of cameras (Kurz, column 11, lines 24-35 and 39-66; column 13, lines 55-67).
Regarding claim 3, Kurtz in view of Shear teaches the image processing apparatus according to claim 2, wherein the specific object is a person, and the one or more processors execute the instructions to estimate the posture of the specific object based on a position of a joint of the person (Kurz, column 11, lines 24-35; column 13, line 57 through column 14, line 20).
Regarding claim 11, Kurtz in view of Shear teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: detect the specific posture movement of the specific object represented by the three-dimensional model based on a movement speed or an amount of deformation of a shape of the specific object (Kurz, column 12, lines 12-28 and 58-67; column 14, lines 5-20 and 47-60; deformation of joint positions resulting from movement and/or posture changes is detected).
Regarding claim 15, Kurtz in view of Shear teaches the image processing apparatus according to claim 1, wherein changing the appearance of the specific object includes adding an object item in a periphery of the specific object (Kurz, column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are added in the periphery of the subject person).
Regarding claim 17, Kurz in view of Shear teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to: determine whether a user has a right corresponding to a display mode, and when the user is determined to have the right correspond to the display mode, generate a virtual viewpoint video of the specific that corresponds to the display mode (Kurz, column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are added to the virtual viewpoint image to indicate the situation, and can comprise various display modes such as virtual object addition, message indication, information superimposition, etc.; Shear, paragraph 276, lines 1-14 and the last 18 lines; paragraph 277, lines 14-32; paragraph 282, the last 19 lines; paragraph 292; the viewpoint video is generated and it is determined whether a user has a right to a display mode).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the rights determination of Shear with the subject person analysis and display of Kurz because this facilitates the vetting and authorization of access to the subject person’s analyzed information, thereby providing a much higher degree of computing resource trustworthiness and increasing the accuracy, reliability, and security of the user’s situation evaluation (see, for example, Shear, paragraph 163; paragraph 178, lines 1-15; paragraph 433, lines 1-12).
Regarding claim 18, Kurz in view of Shear teaches the image processing apparatus according to claim 17, wherein the one or more processors execute the instructions to: determine whether a user owns a non-fungible token (NFT) corresponding to the display mode, and when the user is determined to own the NFT corresponding to the display mode, generate a virtual viewpoint video of the specific object according to the display mode (Kurz, column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; Shear, paragraph 162, lines 1-15; paragraph 1591; where Shear teaches the NFT and video).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the NFT rights determination of Shear with the subject person analysis and display of Kurz because this facilitates the vetting and authorization of access to the subject person’s analyzed information, thereby providing a much higher degree of computing resource trustworthiness and increasing the accuracy, reliability, and security of the user’s situation evaluation (see, for example, Shear, paragraph 163; paragraph 178, lines 1-15; paragraph 433, lines 1-12).
Regarding claim 19, the limitations of this claim substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Regarding claim 20, the limitations of this claim substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Regarding claim 21, Kurz in view of Shear teaches the image processing apparatus according to claim 1, wherein changing the appearance of the specific object includes changing a size of the specific object, changing a color of the specific object, or adding an effect object associated with the specific object (Kurz, column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; an effect object associated with the specific object is added).
Regarding claim 22, Kurz in view of Shear teaches the image processing apparatus according to claim 1, wherein the one or more processors execute the instructions to select a display mode from a plurality of display modes, wherein the appearance of the specific object is changed in different manners in respective display modes (Kurz, column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are added to the virtual viewpoint image to change the appearance corresponding to various display modes such as virtual object addition, message indication, information superimposition, etc.).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kurz, in view of Shear, and further in view of Zhang et al. (U.S. Patent Application Publication No. 2017/0064214), referred herein as Zhang.
Regarding claim 14, Kurz in view of Shear teaches the image processing apparatus according to claim 1, wherein changing the appearance of the specific object includes changing visual cues of the specific object (Kurz, column 12, lines 46-57; column 13, lines 3-10 and 21-32; column 14, lines 10-32; visual cues and/or objects are added to the virtual viewpoint image to indicate the situation).
Kurtz in view of Shear does not teach changing a size or a color of the object.
However, in a similar field of endeavor, Zhang teaches an apparatus configured to obtain a three-dimensional model of an object, obtain information indicating a situation of the object regarding changes in its posture and/or position, and generating a virtual viewpoint image according to the 3D model and the information indicating the situation (figs 3B and 20; paragraph 126; paragraphs 129 and 130; paragraph 134; paragraphs 136-138), wherein the apparatus is further configured to change a size or a color of the object in the virtual viewpoint image in accordance with the situation of the object (paragraph 307, lines 1-15).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the color or size change taught by Zhang with the visual cues of Kurz in view of Shear because this helps the subject person have a better understanding of how the posture needs to be adjusted, thereby increasing the usability and accuracy of the posture analysis procedures (see, for example, Zhang, paragraph 307, the last 10 lines).
Response to Arguments
Applicant’s arguments with respect to the prior art rejections have been fully considered, but they are not persuasive.
On page 7 of the Applicant’s Remarks, with respect to claim 1, the Applicant argues, in summary, that Kurz does not teach the amended limitations regarding generating a viewpoint video. The Examiner respectfully submits that this argument is moot in view of the new grounds of rejection presented above. As noted in the rejection above, Kurtz teaches that the system may comprise images or video and that a plurality of images are obtained over time, but does not explicitly teach that the virtual viewpoint of the object is a video. However, it is respectfully submitted that Kurz in view of Shear teaches these limitations.
On pages 7 and 8 of the Applicant’s Remarks, with respect to claim 1, the Applicant argues, in summary, that 1) Kurz does not teach changing the appearance because the feedback in Kurz, such as adding the obstacle, is not a change of appearance of the specific object, and 2) Kurz does not disclose generating depictions as virtual viewpoint images of the user rendered from a 3D model. The Examiner respectfully disagrees with these arguments.
Regarding the first argument, it is respectfully submitted that the claims are not limited in the manner suggested by the Applicant, and Kurz teaches several ways that the appearance is changed. Using the example discussed in the Remarks regarding Kurz adding the obstacle, Applicant’s claims 21 give three examples of what this change of appearance may comprise, one of which comprises “adding an effect object associated with the specific object” which is taught by Kurz. Thus, to one skilled in the art, and in Applicant’s specification and claims, it is clear that the change of appearance is not limited to a change of appearance of the object itself. Thus Kurz clearly teaches this feature. Additionally, in the one claim that does require a change to the appearance of the object itself (claim 14), such as changing the size or color of the object, Kurz is not relied upon to teach these features, as they are clearly taught by Zhang.
Regarding the second argument, as noted by the Applicant, and as shown in the citations, Kurz teaches, inter alia, evaluating 3D posture and joint positions of the user and rendering images that provide feedback to the user so that they can take real-time corrective actions. The Examiner respectfully submits that this clearly teaches the very broad concept of “generating…depictions as virtual viewpoint images of the user rendered from a three-dimensional model.”
Accordingly, it is respectfully submitted that any potential differences between the claims and the prior art are not currently reflected in the claims. Thus, Kurz in view of Shear teaches the limitations of claim 1, and the rejection is respectfully maintained.
On pages 8 and 9 of the Applicant's Remarks, the Applicant argues that the remaining independent claims are not taught by the prior art for reasons similar to those discussed in regard to claim 1, and that the dependent claims are not taught, insomuch as they depend from claims that are not taught by the prior art. The Examiner respectfully disagrees with these arguments, for the reasons discussed above.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kurillo et al. (Real-Time 3D Avatars for Tele-rehabilitation in Virtual Reality); Medicine Meets Virtual Reality; 2011.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
