DETAILED ACTION
This office action is in response to the amendment filed on 06/13/2025. Claims 1-7, 9, 13, 15, 19, 21, 25, 27, and 31 have been amended. Claims 1-31 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment/Arguments
Applicant’s arguments with respect to claims 1-7, 9, 13, 15, 19, 21, 25, 27, and 31 have been considered but are moot because the arguments do not apply to any of the references being used in the current rejection. Although a new grounds of rejection has been used to address additional limitations that have been added to the claims, a response is deemed necessary for several of applicant’s arguments since Appelgate (US 2021/0321081) will continue to be used to meet several of the claimed limitations.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 6-9, 12-15, 18-21, 24-27 and 30-31 are rejected under 35 U.S.C. 103 as being unpatentable over Appelgate (US 2021/0321081) in view of Shysheya (US 2021/0358197).
Regarding claim 1, Appelgate discloses the following claim limitations: one or more processors, comprising: circuitry (Appelgate, paragraph 48 discloses processing module can include one or more: GPUs, CPUs, TPUs, microprocessors, and/or any other suitable processor),
to use one or more neural networks to generate one or more images of one or more objects in two or more different poses from two or more different points of view (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Appelgate does not explicitly disclose the following claim limitations: where the one or more objects are depicted… comprising different arrangements of one or more joints of the one or more objects, wherein the two or more poses are depicted.
However, in the same field of endeavor Shysheya discloses more explicitly the following: where the one or more objects are depicted… comprising different arrangements of one or more joints of the one or more objects, wherein the two or more poses are depicted (Shysheya, paragraphs 12 and 13 discloses receiving (S101) 3D coordinates of body joint positions of the person defined in a camera coordinate frame, wherein the 3D coordinates of the body joint positions define a pose of the person and a viewpoint of the 2-D image… receiving (S201) a plurality of images of the person in different poses and from different viewpoints).
It would have been obvious to one the ordinary skill in the art at the time of invention to modify the teachings of Appelgate with Shysheya to create neural networks of Appelgate with the camera system of Shysheya.
The reasoning being is to provide a system and method for synthesizing 2-D images of a person or 3-D images of a person (Shysheya, paragraph 9).
Regarding claim 2, Appelgate and Shysheya discloses one or more processors of claim 1, wherein the one or more neural networks are trained using one or more videos of one or more second objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Regarding claim 3, Appelgate and Shysheya discloses one or more processors of claim 1, wherein the one or more neural networks include an encoder to extract features from one or more images of the one or more first objects and encode the features into a latent space, and wherein the extracted features include at least appearance information for a plurality of articulable parts of the one or more first objects and three-dimensional location information for a plurality of joints connecting the plurality of articulable parts (Appelgate, paragraph 83 and Fig. 13 discloses the views (e.g., the set of views corresponding to a light field image) can be extracted from a video (e.g., a stored video, an encoded light field image, a video in the metadata of a thumbnail, etc.) and arranged in a light field image (e.g., a quilt image). The arrangement order can be: specified by the metadata, specified by the encoding scheme (e.g., the arrangement pattern), or otherwise determined).
Regarding claim 6, Appelgate and Shysheya discloses one or more processors of claim 1, wherein the circuitry is further to determine the one or more poses from one or more images of one or more third objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
With regard to claim 7, claim 7 lists all the same elements and features to claim 1 as outlined above. Therefore, the same rationale that was utilized in claim 1 applies equally as well to claim 7.
Regarding claim 8, Appelgate and Shysheya discloses the system of claim 7, wherein the one or more neural networks are trained using one or more videos of one or more second objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Regarding claim 9, Appelgate and Shysheya discloses the system of claim 7, wherein the one or more neural networks include an encoder to extract features from one or more images of the one or more first objects and encode the features into a latent space, and wherein the extracted features include at least appearance information for a plurality of articulable parts of the one or more first objects and three-dimensional location information for a plurality of joints connecting the plurality of articulable parts (Appelgate, paragraph 83 and Fig. 13 discloses the views (e.g., the set of views corresponding to a light field image) can be extracted from a video (e.g., a stored video, an encoded light field image, a video in the metadata of a thumbnail, etc.) and arranged in a light field image (e.g., a quilt image). The arrangement order can be: specified by the metadata, specified by the encoding scheme (e.g., the arrangement pattern), or otherwise determined).
Regarding claim 12, Appelgate and Shysheya discloses the system of claim 7, wherein the one or more processors are further to determine the one or more poses from one or more images of one or more third objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
With regard to claim 13, claim 13 lists all the same elements and features to claim 1 as outlined above. Therefore, the same rationale that was utilized in claim 1 applies equally as well to claim 13.
Regarding claim 14, Appelgate and Shysheya discloses the method of claim 13, wherein the one or more neural networks are trained using one or more videos of one or more second objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Regarding claim 15, Appelgate and Shysheya discloses the method of claim 13, wherein the one or more neural networks include an encoder to extract features from one or more images of the one or more first objects and encode the features into a latent space, and wherein the extracted features include at least appearance information for a plurality of articulable parts of the one or more first objects and three-dimensional location information for a plurality of joints connecting the plurality of articulable parts (Appelgate, paragraph 83 and Fig. 13 discloses the views (e.g., the set of views corresponding to a light field image) can be extracted from a video (e.g., a stored video, an encoded light field image, a video in the metadata of a thumbnail, etc.) and arranged in a light field image (e.g., a quilt image). The arrangement order can be: specified by the metadata, specified by the encoding scheme (e.g., the arrangement pattern), or otherwise determined).
Regarding claim 18, Appelgate and Shysheya discloses the method of claim 13, further comprising: determining the one or more poses from one or more images of one or more third objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
With regard to claim 19, claim 19 lists all the same elements and features to claim 1 as outlined above. Therefore, the same rationale that was utilized in claim 1 applies equally as well to claim 19.
Regarding claim 20, Appelgate and Shysheya discloses the machine-readable medium of claim 19, wherein the one or more neural networks are trained using one or more videos of one or more second objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Regarding claim 21, Appelgate and Shysheya discloses the machine-readable medium of The machine-readable medium of wherein the one or more neural networks include an encoder to extract features from one or more images of the one or more objects and encode the features into a latent space, and wherein the extracted features include at least appearance information for a plurality of articulable parts of the one or more objects and three-dimensional location information for a plurality of joints connecting the plurality of articulable parts Appelgate, paragraph 83 and Fig. 13 discloses the views (e.g., the set of views corresponding to a light field image) can be extracted from a video (e.g., a stored video, an encoded light field image, a video in the metadata of a thumbnail, etc.) and arranged in a light field image (e.g., a quilt image). The arrangement order can be: specified by the metadata, specified by the encoding scheme (e.g., the arrangement pattern), or otherwise determined).
Regarding claim 24, Appelgate and Shysheya discloses the machine-readable medium of claim 19, wherein the instructions if performed further cause the one or more processors to: determine the one or more poses from one or more images of one or more third objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
With regard to claim 25, claim 25 lists all the same elements and features to claim 1 as outlined above. Therefore, the same rationale that was utilized in claim 1 applies equally as well to claim 25.
Regarding claim 26, Appelgate and Shysheya discloses the image generation system of claim 25, wherein the one or more neural networks are trained using one or more videos of one or more second objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Regarding claim 27, Appelgate and Shysheya discloses the image generation system of claim 25, wherein the one or more neural networks include an encoder to extract features from one or more images of the one or more objects and encode the features into a latent space, and wherein the extracted features include at least appearance information for a plurality of articulable parts of the one or more objects and three-dimensional location information for a plurality of joints connecting the plurality of articulable parts (Appelgate, paragraph 83 and Fig. 13 discloses the views (e.g., the set of views corresponding to a light field image) can be extracted from a video (e.g., a stored video, an encoded light field image, a video in the metadata of a thumbnail, etc.) and arranged in a light field image (e.g., a quilt image). The arrangement order can be: specified by the metadata, specified by the encoding scheme (e.g., the arrangement pattern), or otherwise determined).
Regarding claim 30, Appelgate and Shysheya discloses the image generation system of claim 25, wherein the one or more processors are further to determine the one or more poses from one or more images of one or more third objects (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Regarding claim 31, Appelgate and Shysheya discloses the one or more processors of claim 1, wherein the one or more neural networks are used to generate the one or more images of the one or more objects in one or more first poses based, at least in part, on training the one or more neural networks with unlabeled training images of the one or more objects in one or more second poses (Appelgate, abstract, paragraphs 74 and 84 discloses “generating the views using a trained neural network” … “receiving a set of views, encoding the set of views, and displaying the set of views such that they are perceived as a holographic image” i.e., formed images; “light field video can be represented by a single 3D reconstruction with different poses (e.g., representative of the scene pose relative to a point of view)”.
Claim(s) 4, 10, 16, 22 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Appelgate (US 2021/0321081) in view of Shysheya (US 2021/0358197) and in further view of Luengo (US 2024/0156547).
Regarding claim 4, Appelgate and Shyshey discloses the claimed invention as outlined above in claim 3.
Appelgate and Shysheya do not explicitly disclose the following claim limitations: one or more processors of wherein thecircuitry is further to use an implicit function to predict, using features encoded in the latent space and an input viewing direction, color and transparency information for a plurality of points sampled along rays traced for individual pixels of the one or more images.
However, in the same field of endeavor Luengo discloses more explicitly the following: one or more processors of wherein thecircuitry is further to use an implicit function to predict, using features encoded in the latent space and an input viewing direction, color and transparency information for a plurality of points sampled along rays traced for individual pixels of the one or more images (Luengo, paragraphs 17 and 72 discloses the machine learning model includes a feature encoder to predict features from the surgical data for the procedure… wherein the plurality of attributes comprises, a color, a border-type, a transparency, a priority, and an audible sound).
It would have been obvious to one the ordinary skill in the art at the time of invention to modify the teachings of Appelgate and Shysheya with Luengo to create the system of Appelgate and Shysheya as outlined above with predicting features.
The reasoning being is to provide an augmented visualization of the surgical procedure by displaying one or more graphical overlays based on the findings of the machine learning to enhance the surgeon's information (Luengo, abstract).
Regarding claim 10, Appelgate, Shysheya and Luengo discloses the system of claim 9, wherein the one or more processors are further to use an implicit function to predict, using features encoded in the latent space and an input viewing direction, color and transparency information for a plurality of points sampled along rays traced for individual pixels of the one or more images (Luengo, paragraphs 17 and 72 discloses the machine learning model includes a feature encoder to predict features from the surgical data for the procedure… wherein the plurality of attributes comprises, a color, a border-type, a transparency, a priority, and an audible sound). The same motivation that was utilized in claim 4 applies equally as well to claim 10.
Regarding claim 16, Appelgate, Shysheya and Luengo discloses the method of claim 15, further comprising: using an implicit function to predict, using features encoded in the latent space and an input viewing direction, color and transparency information for a plurality of points sampled along rays traced for individual pixels of the one or more images (Luengo, paragraphs 17 and 72 discloses the machine learning model includes a feature encoder to predict features from the surgical data for the procedure… wherein the plurality of attributes comprises, a color, a border-type, a transparency, a priority, and an audible sound). The same motivation that was utilized in claim 4 applies equally as well to claim 16.
Regarding claim 22, Appelgate, Shysheya and Luengo discloses the machine-readable medium of claim 21, wherein the instructions if performed further cause the one or more processors to: use an implicit function to predict, using features encoded in the latent space and an input viewing direction, color and transparency information for a plurality of points sampled along rays traced for individual pixels of the one or more images (Luengo, paragraphs 17 and 72 discloses the machine learning model includes a feature encoder to predict features from the surgical data for the procedure… wherein the plurality of attributes comprises, a color, a border-type, a transparency, a priority, and an audible sound). The same motivation that was utilized in claim 4 applies equally as well to claim 22.
Regarding claim 28, Appelgate, Shysheya and Luengo discloses the image generation system of claim 27, wherein the one or more processors are further to use an implicit function to predict, using features encoded in the latent space and an input viewing direction, color and transparency information for a plurality of points sampled along rays traced for individual pixels of the one or more images (Luengo, paragraphs 17 and 72 discloses the machine learning model includes a feature encoder to predict features from the surgical data for the procedure… wherein the plurality of attributes comprises, a color, a border-type, a transparency, a priority, and an audible sound). The same motivation that was utilized in claim 4 applies equally as well to claim 28.
Claim(s) 5, 11, 17, 23 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Appelgate (US 2021/0321081) in view of Shysheya (US 2021/0358197) and in further view of Issa (US 2014/0092132).
Regarding claim 5, Appelgate and Shyshey discloses the claimed invention as outlined above in claim 3.
Appelgate and Shysheya do not explicitly disclose the following claim limitations: one or more processors of wherein thecircuitry is further to determine the one or more poses based at least in part upon roto-translation data determined for the plurality of joints, as well as parent- child relationship data for pairs of parts among the plurality of articulable parts.
However, in the same field of endeavor Issa discloses one or more processors of wherein thecircuitry is further to determine the one or more poses based at least in part upon roto-translation data determined for the plurality of joints, as well as parent- child relationship data for pairs of parts among the plurality of articulable parts (Issa, paragraph 33 discloses process by which a 3D point in the real-world is brought to the image plane through the camera is called projection, and will be referred to as P. In one embodiment, P can be defined as product between the matrix K, and the 31) roto-translation between the camera and the real-world).
It would have been obvious to one the ordinary skill in the art at the time of invention to modify the teachings of Appelgate and Shysheya with Issa the system of Appelgate and Shysheya as outlined above with determining a pose using roto-translation data.
The reasoning being is to determine the presence in the scene of the object of interest (Issa, [0041]).
Regarding claim 11, Appelgate, Shysheya and Issa discloses the system of claim 9, wherein the one or more processors are further to determine the one or more poses based at least in part upon roto-translation data determined for the plurality of joints, as well as parent-child relationship data for pairs of parts among the plurality of articulable parts (Issa, paragraph 33 discloses process by which a 3D point in the real-world is brought to the image plane through the camera is called projection, and will be referred to as P. In one embodiment, P can be defined as product between the matrix K, and the 31) roto-translation between the camera and the real-world). The same motivation that was utilized in claim 5 applies equally as well to claim 11.
Regarding claim 17, Appelgate, Shysheya and Issa discloses the method of claim 15, further comprising: determining the one or more poses based at least in part upon roto-translation data determined for the plurality of joints, as well as parent-child relationship data for pairs of parts among the plurality of articulable parts (Issa, paragraph 33 discloses process by which a 3D point in the real-world is brought to the image plane through the camera is called projection, and will be referred to as P. In one embodiment, P can be defined as product between the matrix K, and the 31) roto-translation between the camera and the real-world). The same motivation that was utilized in claim 5 applies equally as well to claim 17.
Regarding claim 23, Appelgate, Shysheya and Issa discloses the machine-readable medium of claim 21, wherein the instructions if performed further cause the one or more processors to: determine the one or more poses based at least in part upon roto-translation data determined for the plurality of joints, as well as parent-child relationship data for pairs of parts among the plurality of articulable parts (Issa, paragraph 33 discloses process by which a 3D point in the real-world is brought to the image plane through the camera is called projection, and will be referred to as P. In one embodiment, P can be defined as product between the matrix K, and the 31) roto-translation between the camera and the real-world). The same motivation that was utilized in claim 5 applies equally as well to claim 23.
Regarding claim 29, Appelgate, Shysheya and Issa discloses the image generation system of claim 27, wherein the one or more processors are further to determine the one or more poses based at least in part upon roto- translation data determined for the plurality of joints, as well as parent-child relationship data for pairs of parts among the plurality of articulable parts (Issa, paragraph 33 discloses process by which a 3D point in the real-world is brought to the image plane through the camera is called projection, and will be referred to as P. In one embodiment, P can be defined as product between the matrix K, and the 31) roto-translation between the camera and the real-world). The same motivation that was utilized in claim 5 applies equally as well to claim 29.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JERRY T JEAN BAPTISTE whose telephone number is (571)272-6189. The examiner can normally be reached Monday-Friday 9-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at 571-272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JERRY T JEAN BAPTISTE/ Primary Examiner, Art Unit 2481