Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1,4-5,8-9, 11, 17 and 20 is/are rejected under 35 U.S.C. 102(a) as being anticipated by Gallaway (WO 2021/011070 A1) in view of Mensink (US 2014/0029389 A1).
Regarding claim 1, Gallaway teaches a computer-implemented method for generating representations of scenes (Figure 5, #522), the method comprising:
assigning each image (Fig. 1, #202,#204) included in a set of images of a scene to at least one cluster of images in a plurality of clusters (paragraph [0008], lines 21-23, fig. 2 #208) based on a camera pose associated with the image; (paragraph [0004], lines 24-28) and
performing one or more operations to generate a corresponding three-dimensional (3D) representation of the scene based on one or more images assigned to the cluster. ("The cluster of images are then used to get a 3D point cloud", paragraph [0010])
Gallaway fails to teach performing one or more operations to generate, for each cluster included in the plurality of clusters, a corresponding three-dimensional (3D) representation of the scene based on one or more images assigned to the cluster.
However, Mensink teaches performing one or more operations to generate, for each cluster included in the plurality of clusters, a representation of the scene based on one or more images assigned to the cluster (paragraphs [0081], [0082]). Mensink describes generating multidimensional representations for each of the training images and generating cluster representations by averaging the multidimensional representations associated with each cluster. This representation generation is analogous generating a representation for each cluster in the plurality of clusters. Mensink is considered analogous to the claimed invention as it is in the same field of image processing and clustering. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the generation of a representation for each cluster in a plurality of clusters from Mensink with the teachings of Gallaway to allow for comparison between different clusters, which improves the assignment and classification of clusters.
Computer-readable media (CRM) claim 11 is drawn to the CRM corresponding to the method of using same as claimed in claim 1. Therefore apparatus CRM claim 11 corresponds to method claim 1, and is rejected for the same reasons of anticipation as used above.
Apparatus claim 20 is drawn to the apparatus corresponding to the method of using same as claimed in claim 1. Therefore apparatus claim 20 corresponds to method claim 1, and is rejected for the same reasons of anticipation as used above.
Regarding claim 4, Gallaway in view of Mensink teaches the computer-implemented method of claim 1. Gallaway further teaches wherein assigning each image included in the set of images to the at least one cluster of images comprises:
computing a view ray associated with each image included in the set of images based on the camera pose associated with the image; (paragraph [0013]) and
assigning each image included in the set of images to the at least one cluster of images based on one or more points that are closest to the view rays that are computed. (paragraph [0008])
Regarding claim 5, Gallaway in view of Mensink teaches the computer-implemented method of claim 1. Gallaway further teaches wherein each corresponding 3D representation comprises at least one of a neural radiance field (NeRF), a signed distance function (SDF), a set of points, a set of surfels (Surface Elements), a set of Gaussians, a set of tetrahedra, or a set of triangles. (paragraphs [0006],[0009], and [0032])
Regarding claim 8, Gallaway in view of Mensink teaches the computer-implemented method of claim 1. Gallaway further teaches further comprising:
rendering a plurality of images based on at least two of the corresponding
3D representations of the scene; and
performing one or more operations to interpolate the plurality of images to
generate an interpolated image. (paragraph [0013])
Regarding claim 9, Gallaway in view of Mensink teaches the computer-implemented method of claim 1. Gallaway further teaches further comprising adding at least one image included in the set of images to at least one cluster included in the plurality of clusters based on a view direction associated with the at least one image (paragraphs [0004] and [0008]).
Computer-readable media (CRM) claim 17 is drawn to the CRM corresponding to the method of using same as claimed in claim 9. Therefore apparatus CRM claim 17 corresponds to method claim 9, and is rejected for the same reasons of anticipation as used above.
Claim(s) 2, 3, and 6, 12, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gallaway in view of Mensink and in further view of Andrinov (WO 2022/079008 A1).
Regarding claim 2, Gallaway in view of Mensink teaches the computer-implemented method of claim 1. Gallaway further teaches wherein assigning each image included in the set of images to the at least one cluster of images comprises: computing a point associated with each image based on the camera pose associated with the image; (paragraphs [0004], [0008]).
Gallaway in view of Mensink fails to teach determining one or more cluster centers based on the point associated with each image; and assigning each image to the at least one cluster based on distances from the point associated with the image to one or more cluster centers.
However, Andrivon teaches determining one or more cluster centers based on the point associated with each image; and assigning each image to the at least one cluster based on distances from the point associated with the image to one or more cluster centers. (paragraph [0031]).Andrivon associates sectors around the point cloud based on angle or distance from the center of the reconstructed point cloud. A point cloud and a sector around the point cloud can be considered a cluster because they are a collection of three-dimensional coordinates associated based on some set of features. Therefore the calculation of the distance from the center of the point cloud to associate sectors is analogous to the association of clusters based on distances from the point associated with the image to one or more cluster centers.
Andrivon is considered analogous to the claimed invention since it is in the same field of three-dimensional reconstruction and computer graphics. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to incorporate the teachings of Andrivon with the teachings of Gallaway in view of Mensink to determine a cluster reference point.
Computer-readable media (CRM) claim 12 is drawn to the CRM corresponding to the method of using same as claimed in claim 2. Therefore apparatus CRM claim 12 corresponds to method claim 2, and is rejected for the same reasons of anticipation as used above.
Regarding claim 3, the combination of Andrivon and Gallaway in view of Mensink teach the computer-implemented method of claim 2 as described above. Gallaway further teaches wherein the point associated with each image is located at either a predefined distance or a computed distance from a camera that captured the image (paragraph [008]). Gallaway describes mapping image location with respect from to their distances from the camera.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Gallaway in view of Mensink and Andrinov to map images to an associated point.
Regarding claim 6, Gallaway in view of Mensink teaches the computer-implemented of claim 1.
Gallaway in view of Mensink fails to teach further comprising rendering an image based on at least one corresponding 3D representation of the scene that is located closer to a viewer with respect to a distance metric than at least one other corresponding 3D representation of the scene.
However, Andrinov teaches further comprising rendering an image based on at least one corresponding 3D representation of the scene that is located closer to a viewer with respect to a distance metric than at least one other corresponding 3D representation of the scene. (abstract).
Therefore it would have been obvious to one of ordinary skill in the art to incorporate the teachings of Andrinov with the teachings of Gallaway.
Computer-readable media (CRM) claim 14 is drawn to the CRM corresponding to the method of using same as claimed in claim 6. Therefore apparatus CRM claim 14 corresponds to method claim 6, and is rejected for the same reasons of anticipation as used above.
Claim(s) 7, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gallaway in view of Mensink and in further view of Chang (US 20040222987 A1).
Gallaway in view of Mensink teaches the computer implemented method of claim 1. Gallaway in view of Mensink fails to teach further comprising rendering an image based on four corresponding 3D representations of the scene and barycentric coordinates associated with a viewer within a tetrahedron formed by the four corresponding 3D representations.
However, Chang teaches further comprising rendering an image based on four corresponding 3D representations of the scene and barycentric coordinates associated with a viewer within a tetrahedron formed by the four corresponding 3D representations (Paragraphs [0119], [0127], figures 15, 18a-d). Chang generates a synthetic view (representation) from 3 views and barycentric coordinates associated with the user's view. Chang also suggests in paragraph 119 to use four anchor views consisting of a tetrahedron.
Chang is considered analogous to the claimed invention as it is in the same field of three-dimensional view reconstruction and computer graphics.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to incorporate the teachings of Chang with the teachings of Gallaway in view of Mensink to improve three-dimensional scene representation generation.
Computer-readable media (CRM) claim 15 is drawn to the CRM corresponding to the method of using same as claimed in claim 7. Therefore apparatus CRM claim 15 corresponds to method claim 7, and is rejected for the same reasons of anticipation as used above.
Claim(s) 10, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gallaway in view of Mensink and in further view of Kutliroff (US 20180018805 A1).
Gallaway in view of Mensink teaches the computer-implemented method of claim 1. Gallaway in view of Mensink fails to teach further comprising, prior to assigning each image included in the set of images to the at least one cluster of images, performing one or more operations to segment out one or more predefined classes of objects within each image included in the set of images.
However, Kutliroff teaches further comprising, prior to assigning each image included in the set of images to the at least one cluster of images, performing one or more operations to segment out one or more predefined classes of objects within each image included in the set of images (Fig. 14, paragraph [0056]). Kutliroff is considered analogous to the claimed invention as it is in the same field of three-dimensional reconstruction and computer graphics.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to incorporate the teachings of Kutliroff with Gallaway in view of Mensink to implement segmentation.
Computer-readable media (CRM) claim 18 is drawn to the CRM corresponding to the method of using same as claimed in claim 10. Therefore apparatus CRM claim 18 corresponds to method claim 10, and is rejected for the same reasons of anticipation as used above.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gallaway in view of Mensink and Andrinov and in further view of Jiang (CN 107564095 A).
Regarding claim 13, Gallaway in view of Mensink and Andrinov teaches the one or more non-transitory computer-readable media of claim 12.
Gallaway fails wherein determining the one or more cluster centers comprises performing one or more k-means clustering operations based on the points associated with the set of images.
However, Jiang teaches wherein determining the one or more cluster centers comprises performing one or more k-means clustering operations based on the points associated with the set of images (page 3, step A1). Jiang describes a k-means clustering approach based on the pixel luminance and coordinate information, which is analogous to k-means clustering based on the points associated with the set of images. Jiang doesn’t directly describe determining cluster centers but, by definition, the k-means clustering algorithm calculates the center of a cluster.
Jiang is considered analogous to the claimed invention as it is in the same field of computer graphics. Therefore, it would have obvious to one of ordinary skill in the art, before the effective filing date, to incorporate the teachings of Jiang with Gallaway in view of Mensink and Andrinov to specify the clustering algorithm.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gallaway in view of Mensink and in further view of Mammou (US 2019/0311500 A1).
Regarding claim 16, Gallaway in view of Mensink teaches the one or more non-transitory computer-readable media of claim 11.
Gallaway in view of Mensink fails to teach wherein one of the corresponding 3D representations of the scene is associated with a data structure that identifies a region of the scene that is larger in size than another region of the scene that is identified by another data structure associated with another one of the corresponding 3D representations of the scene.
However, Mammou teaches wherein one of the corresponding 3D representations of the scene is associated with a data structure that identifies a region of the scene that is larger in size than another region of the scene that is identified by another data structure associated with another one of the corresponding 3D representations of the scene. (paragraph [0158], first two sentences). Mammou describes sorting patches based on the area of their bounding boxes. Both the patches and bounding boxes of Mammou represent regions of the scene. The process of sorting based on size described in Mammou necessitates the identification of a region that is larger than another.
Mammou is considered analogous to the clamed invention as it is in the same field of three-dimensional computer graphics. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to incorporate the teachings of Mammou into Gallaway in view of Mensink in order to sort regions of a scene based on size.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gallaway in view of Mensink and Kultiroff and in further view of Lopez (US 9609307 B1).
Regarding claim 19, Gallaway in view of Mensink and Kultiroff teach the one or more non-transitory computer-readable media of claim 18.
Gallaway in view of Mensink and Kultiroff fail to teach wherein the one or more predefined classes of objects include at least one class of objects that is able to move within the scene.
However, Lopez teaches wherein the one or more predefined classes of objects include at least one class of objects that is able to move within the scene. (col. 18, lines 40-43). Lopez describes identification of stationary and moving objects. This is analogous to segmenting predefined classes wherein one class of object is able to move.
Lopez is considered analogous to the claimed invention as it is in the same field of three-dimensional reconstruction and computer graphics. Therefore it would have been obvious to one of ordinary skill in the art to modify the teachings of Gallaway in view of Mensink and Kultiroff with the teachings of Lopez to identify a more specific class of objects, that being moving objects.
Response to Arguments
Applicant's arguments filed December 19th 2025 have been fully considered but they are not persuasive. Applicant argues on pages 8 and 9 that Mensink “does not teach or suggest that the set of current sample images comprise images for the same scene”. Examiner does not find this argument to be persuasive. The citations of Mensink are not directed towards scene representations, but instead techniques of clustering. Gallaway is relied upon as the primary reference to teach assigning images of a scene to a cluster, and to generate a 3D representation of the scene using the cluster information. However, Gallaway fails to generate a representation for each cluster. Mensink, however, does generate a representation of the data, for each cluster. Applicant states, on page 9, “Mensink would have to disclose assigning each current sample image included in a set of current sample images of a scene to at least one cluster of images, and generating, for each cluster included in the plurality of clusters, a corresponding multidimensional cluster representation of the scene based on one or more sample images assigned to the cluster”. Examiner respectfully disagrees because the primary reference, Gallaway, teaches a three-dimensional (3D) representation of the scene based on one or more images assigned to the cluster (paragraphs [0008], [0010], [0012]). Therefore, examiner maintains the previous rejection and the position that it would have been obvious to one of ordinary skill in the art to combine the teachings of Gallaway and Mensink to apply the 3D representation generation of the scene of Gallaway with the multi-dimensional representation generated for each cluster of Mensink.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
S. M. Sawyer, K. Ni and N. T. Bliss, "Cluster-based 3D reconstruction of aerial video," 2012 IEEE Conference on High Performance Extreme Computing, Waltham, MA, USA, 2012, pp. 1-6, doi: 10.1109/HPEC.2012.6408681.
D. Zhang, R. Zheng and G. Yang, "Image clusters based 3D virtual tour schema," 2014 IEEE 5th International Conference on Software Engineering and Service Science, Beijing, China, 2014, pp. 641-644, doi: 10.1109/ICSESS.2014.6933650.
X. Wang, Y. Li, C. Wang and Y. Qi, "Distributed Generation of Large-Scale 3D Dense Point Cloud for Accurate Multi-View Reconstruction," 2019 International Conference on Virtual Reality and Visualization (ICVRV), Hong Kong, China, 2019, pp. 82-86, doi: 10.1109/ICVRV47840.2019.00022.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aidan W McCoy whose telephone number is (571)272-5935. The examiner can normally be reached 8:00 AM-5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AIDAN W MCCOY/Examiner, Art Unit 2611
/TAMMY PAIGE GODDARD/Supervisory Patent Examiner, Art Unit 2611