DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Received 05/06/2026
Claim(s) 1-20 is/are pending.
Claim(s) 2-4, 6, and 20 has/have been amended.
The objections to the Specification is/are maintained in view of the amendments received 05/06/2026.
The 35 U.S.C § 103 rejection to claim(s) 1-20 have been fully considered in view of the amendments received on 05/06/2026 and are fully addressed in the prior art rejection below.
Response to Arguments
Received 05/06/2026
Regarding dependent claims 2-8, 10-14, and 16-20:
Applicant’s arguments (Remarks, Page 9: ¶ 3 to Page 10: ¶ 1), filed 05/06/2026, with respect to the rejection(s) of claim(s) 2-8, 10-14, and 16-20 under 35 U.S.C § 103 have been fully considered and are persuasive due the dependency upon claims 1, 9, and 15 respectively. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made.
Applicant's arguments filed 05/06/2026 have been fully considered but they are not persuasive; as expressed below.
Regarding independent claim(s) 1, 9, and 15:
Applicant argues (Remarks, Page 7, ¶ 3), that “… the first reference, Chernov, generally describes generating a 3D mesh using depth maps and corresponding images captured from multiple perspectives, and each pixel of a depth map is projected to populate a 3D grid, and the corresponding images are later used to determine texture. See Chernov ¶¶[0058], [0067]-[0069], [0127]-[0128]; Fig. 2A-2B, 13. However, Chernov does not teach ‘project[ing] the region to generate one or more second images depicting the 3D object from a second perspective that is a different perspective from the first perspective,’ nor ‘determin[ing] one or more differences between the one or more second images and one or more third images that captures the 3D object from the second perspective.’ ”.
The Examiner disagrees. Applicant’s arguments fail to the rejection as a whole. Wherein, the combination does not rely on the teaches of Chernov et al. (US PGPUB No. 20170046868 A1) alone. Such that, the argued subject matter is addressed in more detail in view of Jin et al. (US PGPUB No. 20130124148 A1) below.
Applicant argues (Remarks, Page 8, ¶ 1), that “…the second reference, Jin, also describes generating a 3D mesh using images captured from different viewpoints. A region in one image is mapped to corresponding regions in other images to compare depth values, and consistent depth values are used to form the final 3D mesh. See Jin at ¶¶[0049], [0061], [0063]-[0073]; Fig. 5-6. However, Jin does not disclose project[ing] a region of a 3D mesh to generate another image from a different perspective as claimed. At most, Jin discloses projecting portions of one image to identify corresponding portions in other images.”
The Examiner disagrees. Applicant’s arguments fail to view the teachings of Jin et al., wherein Jin et al. teaches “… depth map module 120 may project a vector from the pixel position to the relative camera position of the image” (Jin; [¶ 0065]; moreover, “Each depth position along the projected vector may represent a 3D location in space, between the actual pixel position on the image and the relative camera position” [¶ 0066]), the pixel position corresponding to a three-dimensional pixel position (Jin; “A depth map may be a set of data in which each data value corresponds to a depth, or 3D position, for a corresponding pixel in an image” [¶ 0063]) further corresponding to a three-dimensional mesh (Jin; “… depth map module 120 may create a tessellated mesh representation of a 3D model that may approximate the 3D surface of an imaged object” [¶ 0063]). Applicant fails to view under BRI (MPEP; [2111]) that a pixel corresponds to a region (e.g. a region the size of one or more pixels). Jin et al. further teaches image regions from one or more viewpoints (Jin; “… a window-based search algorithm for computing depth maps from multiple digital images. The window-based search algorithm may create depth maps only for portions of an image that can be accurately matched to other, neighboring images. As a result, each depth map may contain one or more incomplete surfaces for an imaged object. For example, incomplete surfaces may exist for image regions which have effects that are difficult to match, such as occlusions, low-textured regions, and/or oblique surfaces. However, such effects may typically occur at different regions on an object in different images” [¶ 0063]) and further defines the regions as a grouping of pixels related to a surface of an object (Jin; “For a pixel p in a reference view, depth map module 120 may define an mxm (where m represents a number of pixels) square window (e.g., a reference window) centered around the pixel. For the pixel p inside the bounding volume of an object in a reference view, depth map module 120 may project a vector from the pixel position to the relative camera position of the image” [¶ 0065]). In other words, Applicant fails to view the m time m element (implicitly corresponding to a matrix of pixels which corresponds to a region) that is within the boundaries of the three-dimensional object. The depth map module generates a three-dimensional mesh of pixel (regions) of an object’s surface and projects in the form of a vector (e.g. light ray) to a camera (Jin; “… depth map module 120 may project a vector from the pixel position to the relative camera position of the image” [¶ 0065]; wherein, “Each depth position along the projected vector may represent a 3D location in space, between the actual pixel position on the image and the relative camera position. For each depth position, depth map module 120 may project a vector from the 3D location represented by the depth position to each neighboring view of the object. Using the projected vectors, depth map module 120 may determine a corresponding pixel location in each of the neighboring views” [¶ 0066]). In other words, the depth module creates pixel regions/segments that form mesh structures that are three-dimensional that can be projected to other mesh structures that are created in other image viewpoints to establish a final three-dimensional structure of the object (Jin; “Depth map module 120 may select neighboring views to a reference view based on the relative camera positions for each view. For camera positions around a ring encircling the object or distributed on a hemisphere, neighboring views may be selected based on an angular distance between the optical axes of two views … Depth map module 120 may compute a normalized cross-correlation (NCC) score for each neighboring window by comparing each neighboring window to the reference window centered around the pixel in the reference view. In some embodiments, a high NCC score may indicate that image data of a neighboring window is similar to image data of the reference window, and, therefore, may indicate that the two views represented by the windows are likely to include the same surface area of a textured object. In some embodiments, a low NCC score may indicate poor correlation between the neighboring and reference windows, and, therefore, may indicate that the two views represented by the windows are not likely to include the same surface area of a textured object” [¶ 0067-0068]; moreover, “A depth value for a depth position of pixel p in a reference view may be considered valid if a certain number of neighboring views are determined to match the reference view. For example, in some embodiments, a depth value for a depth position of pixel p may be considered valid if at least two neighboring views are determined to match the reference view (e.g., the neighboring views have NCC scores above a certain threshold, as described above)” [¶ 0069]; such that, “merging the depth maps of the digital images into a tessellated mesh representation of the 3D surface of the imaged object. As described above, a depth map corresponding to a single image may contain one or more incomplete surfaces for an imaged object” [¶ 0073]). Thus, Applicant fails to view the teachings of a “… tessellated mesh representation creating by merging the multiple depth maps may represent an approximation of the 3D surface of the imaged object” (Jin; [¶ 0074]).
Since Jin et al. is in combination with Chernov et al., even more support and context is provided by Chernov et al. regarding the creation and further processing of a depth map and mesh structures (as taught by Jin et al.). Wherein, Chernov et al. teaches that “During the post-processing stage 22, the device 106 first generates a depth map 205 regarding the object 102 included in the images 201. Since the images 201 are captured in different camera orientations, the depth map 205 may include information about a depth of the object 102 in each camera orientation. Then, the device 106 reconstructs a representation of a 3D mesh 211 regarding the object 102 by fusing the depth maps 205 respectively corresponding to the camera orientations” (Chernov; [¶ 0058]; the post-processing [id.] is based on image data [¶ 0057]). Additionally, Applicant fails to view the teachings of camera position corresponding to a new image (Chernov; [¶ 0076]). Wherein, “The key points of the map are projected onto the new image 404, and the camera position corresponding to the new image 404 is estimated by using a Direct Image Alignment algorithm” (Chernov; [¶ 0078-0079]; wherein, “… a camera position corresponding to the new image 404 is estimated based on key points of a map stored in the map DB 304” [id.], and “All sequential images 306 captured by a camera (i.e., the scanning module) of the device 106 are used together with maps stored in the map DB 304 to determine which frame (or image) corresponds to a key frame for tracking and 3D reconstruction” [¶ 0073-0074]). Thus, the combination teaches the projecting of key points (e.g. pixels) onto a new image (Chernov; [¶ 0058 and ¶ 0078-0079]) that are based on a region (i.e. pixels) within a 3D mesh (Jin; [¶ 0063 and ¶ 0065-0066]).
Also, Applicant fails to consider the teachings of sparse image alignment (see Fig. 4 and Fig. 5) within Chernov et al. (Chernov; “ In operation 502, patches of all filtered points are combined in one feature vector. In operation 504, the feature vector is tracked by using the LKT in the new image 404. Then, a camera position 506 of the new image 404 is obtained from the LKT” [¶ 0090-0091]; such that, “The key points of the map are projected onto the new image 404, and a position of each key point in the new image 404 is refined by using a Lucas Kanade tracker (LKT or KLT). The LKT tracks a patch (rectangular area) around a projection of the key point by using gradient descent. The LKT is disclosed in, for example, Simon Baker and lain Matthews, “Lucas-kanade 20 years on: A unifying framework”, International journal of computer vision, 56(3):221-255, 2004” [¶ 0079]); as well as, the teachings of sparse feature alignment (see Fig. 4 and Fig. 6) within Chernov et al. (Chernov; “In operation 602, a patch is independently created for each feature point. In operation 604, each patch is tracked by using the LKT in the new image 404. An updated position of each feature point in the new image 404 is obtained in operation 606” [¶ 0093]). The teachings of the sparse image alignment and sparse feature alignment provide further detail regarding the steps involved when generating and updating a reconstructed or synthesized image based on an indicated viewpoint.
Therefore, Applicant’s arguments above are not persuasive.
Lastly, the claim language is silent regarding “project[ing] a region of a 3D mesh to generate another image from a different perspective” as argued by Applicant. Although, these limitations are presented within the claims (claim 1; “… project the region to generate one or more second images depicting the 3D object from a second perspective that is a different perspective from the first perspective”), the manner in which Applicant argues them are altered. Please be mindful of this, since the meaning of the subject matter and the language of the claimed subject matter, may not always be the equivalent.
Applicant argues (Remarks, Page 8, ¶ 2), that “The third reference, Trenholm, describes using a neural network to classify objects and is cited to purportedly teach the claimed neural network. See Trenholm at ¶¶[0053] and [0058] However, the neural network in Trenholm is used for object classification, not for projecting regions of a 3D mesh or refining a 3D mesh representation. The Office's assertion that any neural network could be substituted to perform the claimed elements is improper as neural networks are trained to do specific tasks. Neural networks cannot simply be substituted to generate a specific output without the proper training and refining.”
The Examiner disagrees. Applicant’s arguments fail to view the broadness of the claim construction and limitations thereof. Wherein, Applicant’s claim 1 recites “… one or more neural networks”, which fails to indicate a specific neural network structure or type. Additionally, one of ordinary skill in the art at the effective filling date of the invention would understand the capabilities and programmability of creating and using a neural network to achieve one or more sought after steps. Still further, claim 1 fails to incorporate limitations that indicate a specific or specialized training/trained neural network to be used to perform one or more steps.
Therefore, Applicant’s arguments above are not persuasive.
Lastly, no general assertion that any neural network can or could be used to perform claimed elements has been made. The rejection made by the combination simply modifies and/or incorporates existing computer performed logic and steps with the machine learning (i.e. neural network) performed logic and steps as taught by Trenholm et al. (US PGPUB No. 20190138786 A1) (Trenholm; [¶ 0064-0066]). Wherein, the mere recitation of a neural network fails incorporate the level and detail of mean as sought by Applicant, and more importantly the limitation of the neural network is claimed in manner that gives it minimal weight based on the body of the claim. Applicant fails to view the Specification of the Disclosure is also unsuccessful at indicating a more limiting neural network configuration than as taught by the applied prior art (Spec.; “Machine learning models created by training engine 122 can include any technically feasible form of machine learning model. For example, the machine learning models may include recurrent neural networks (RNNs), convolutional neural networks (CNNs), deep neural networks (DNNs), deep convolutional networks (DCNs), deep belief networks (DBNs), restricted Boltzmann machines (RBMs), long-short-term memory (LSTM) units, gated recurrent units (GRUs), generative adversarial networks (GANs), self-organizing maps (SOMs), and/or other types of artificial neural networks or components of artificial neural networks. In another example, the machine learning models may include functionality to perform clustering, principal component analysis (PCA), latent semantic analysis (LSA), Word2vec, and/or another unsupervised learning technique. In a third example, the machine learning models may include regression models, support vector machines, decision trees, random forests, gradient boosted trees, naïve Bayes classifiers, Bayesian networks, hierarchical models, and/or ensemble models” [¶ 0023]).
Applicant argues (Remarks, Page 8, ¶ 3), that “The fourth reference, Kawahara, describes receiving images from cameras at different viewpoints, separating foreground from background, and generating a 3D model of the foreground. A simulation engine then generates a synthetic image corresponding to one of the viewpoints, which is compared with an actual camera image to update the simulation. See Kawahara ¶¶[0001], [0004], [0028], and [0032]. However, the generated image in Kawahara is not produced by projecting a region of a 3D mesh representation as claimed. ”.
The Examiner disagrees. Applicant’s arguments fail to view the combination as a whole. The subject matter regarding projecting a region of a 3D mesh is mentioned in more detail in relation with the teachings of Jin et al. and Chernov et al., as addressed above. Wherein, Kawahara (US PGPUB No. 20190335162 A1) is not relied upon to teaches the subject matter of a three-dimensional mesh.
Therefore, Applicant’s arguments above are not persuasive.
Applicant argues (Remarks, Page 8, ¶ 4-5), that “The cited references fail to teach the claimed elements individually, and as a result Applicant submits that the proposed combination also does not render the claims obvious. For example, the neural network in the third reference is directed to object classification rather than 3D mesh generation or refinement, and its incorporation would not result in the claimed invention.
Therefore, for at least the foregoing reasons, Applicant respectfully submits that the proposed combination of Chernov, Jin, Trenholm, and/or Kawahara, either alone or in combination, does not teach or suggest claim 1. Applicant respectfully submits that claim 1 is allowable over Chernov, Jin, Trenholm, and/or Kawahara and respectfully requests withdrawal of the rejection.”
The Examiner disagrees. Applicant’s arguments fail to (1) view the rejection as a whole, and (2) the teachings of the applied prior art. Wherein, the subject matter regarding using one or more neural networks is addressed in more detail above. Even further, Applicant’s arguments fail to view the teachings of “AI module 118 may be configured to: train a neural network based on (i) multi-orientation 3D representations of the object of interest generated from the reconstructed 3D image and/or (ii) multi-view 2D projections of the object of interest generated from the reconstructed 3D image” that are more than just classification (Trenholm; [¶ 0064]). In other words, although classification is a feature of the used neural network, the neural network performs serval additional operations (e.g. segmentation or feature detection) (Trenholm; [¶ 0074-0076 and ¶ 0080-0082]; moreover, “… the AI module 118 of system 100 is configured to receive a reconstructed 3D input image containing the object of interest, segment the 3D image to extract the 3D object of interest, and generate a prediction of class label for the object of interest. At block 801, the 3D reconstructed image (exported from reconstruction module 120 at block 603) is imported to the AI module 118. At block 802, the 3D image is segmented using a segmentation algorithm, as described herein, to isolate and extract the object of interest. Once segmentation is complete, system 100 implementing method 800 can operate in two modes, a 3D mode and a 2D mode, as described in reference to FIG. 7” [¶ 0073]) that can be applied to the image segmentations and classification of object of Jin et al. (Jin; “… in FIG. 7 may include converting the 3D constraints into Bezier curves. Each of the 3D shape constraints created at 702 may be represented as a set of line segments. A set of line segments may not be a representation of a 3D constraint that is easy, or intuitive, for a user to manipulate in order to change the shape of a 3D model. For example, moving the position of a 3D constraint represented by multiple line segments may require a user to select and move several of the line segments, which may be tedious and/or confusing for a user” [¶ 0079-0080]; and “… image analyzer 110 may determine Gaussian models for the foreground and background areas of the image. The Gaussian models may be applied to a whole image and each pixel of the image may be classified as either a foreground pixel or background pixel. Image analyzer 110 may assume that an imaged object exists in the foreground area of an image and, thus, may identify foreground pixels as the imaged object” [¶ 0056 and ¶ 0059-0060]) and extracted points of Chernov et al. (Chernov; “… features from accelerated segment test (FAST) algorithm using corner detection may be used to extract key points from the first image 300” [¶ 0073]).
Therefore, Applicant’s arguments above are not persuasive.
Applicant’s arguments (Remarks, Page 9, ¶ 1-2), filed 05/06/2026, with respect to the rejection(s) of claim(s) 9 and 15 under 35 U.S.C § 103 have been fully considered but they are not persuasive due to claim 9’s and claim 15’s similarity to claim 1. Therefore, the rejection is maintained for reasons as addressed above.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words. The form and legal phraseology often used in patent claims, such as "means" and "said," should be avoided. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, "The disclosure concerns," "The disclosure defined by this invention," "The disclosure describes," etc.
The abstract of the disclosure is objected to because “… embodiment …” Correction is required. See MPEP § 608.01(b).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-15, and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chernov et al., US PGPUB No. 20170046868 A1, hereinafter Chernov, in view of Jin et al., US PGPUB No. 20130124148 A1, hereinafter Jin, in view of Trenholm et al., US PGPUB No. 20190138786 A1, hereinafter Trenholm, and further in view of Kawahara, US PGPUB No. 20190335162 A1, hereinafter Kawahara.
Regarding claim 1, Chernov discloses one or more processors (Chernov; processor(s) [¶ 0048 and ¶ 0159-0160], as illustrated within Fig. 18), comprising:
circuitry to use one or more neural networks (processor(s) [as addressed above], comprises circuitry to use one or more processes [¶ 0160-0161, ¶ 0182, and ¶ 0189]) to:
project one or more first images depicting a three-dimensional (3D) object from a first perspective onto one or more 3D mesh representations (Chernov; the processes [as addressed above] (configured) to project one or more 1st images depicting a 3D object from an implicit 1st perspective (given images of a capture position/angle) onto one or more 3D mesh representations [¶ 0057-0058], as illustrated within Fig. 2A; moreover, mapping textures on to a surface mesh [¶ 0066-0069]; wherein, high quality 3D model reconstruction may be provided via accurate surface reconstruction and texture mapping [¶ 0070]);
determine a region within the one or more 3D mesh representations corresponding to the first perspective (Chernov; the processes [as addressed above] (configured) to determine a region/surface within the one or more 3D mesh representations corresponding to the implicit 1st perspective (given images/textures are based on the capture position/angle) [¶ 0127-0130]; wherein, a surface corresponds to a number of polygons [¶ 0124-0125]; and wherein, assigning mesh faces as visible or invisible from a camera position [¶ 0134-0135 and ¶ 0139-0140]);
determine one or more differences (Chernov; the processes [as addressed above] (configured) to determine one or more differences between images [¶ 0128-0130]); and
modify the one or more 3D mesh representations based, at least in part, on the one or more differences (Chernov; the processes [as addressed above] (configured) to modify the one or more 3D mesh representations based (at least in part) on the one or more differences [¶ 0128-0130]).
Chernov fails to disclose circuitry to use one or more neural networks;
project the region to generate one or more second images depicting the 3D object from a second perspective that is a different perspective from the first perspective; and
determine one or more differences between the one or more second images and one or more third images that captures the 3D object from the second perspective.
However, Jin teaches project the region to generate one or more second images depicting the 3D object from a second perspective that is a different perspective from the first perspective (Jin; project the region [¶ 0063-0066] to generate one or more 2nd images depicting the 3D object from a 2nd perspective that is a different perspective from the 1st perspective [¶ 0040-0042]; wherein, image data is associated with mesh construction [¶ 0044-0045] and image analysis [¶ 0051-0052, ¶ 0055, and ¶ 0059-0060]; moreover, mesh tessellation [¶ 0035 and ¶ 0073-0074]);
determine one or more differences (Jin; determine one or more differences between images [¶ 0068-0070]); and
modify the one or more 3D mesh representations based, at least in part, on the one or more differences (Jin; modify the one or more 3D mesh representations based (at least in part) on the one or more differences [¶ 0072-0074]; moreover, 3D surface approximation [¶ 0045 and ¶ 0063]; wherein, reconstructing a 3D model [¶ 0083-0086]).
Chernov and Jin are considered to be analogous art because both pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov, to incorporate to: project the region to generate one or more second images depicting the 3D object from a second perspective that is a different perspective from the first perspective; determine one or more differences; and modify the one or more 3D mesh representations based, at least in part, on the one or more differences (as taught by Jin), in order to provide an improved modeling while reducing system resources (Jin; [¶ 0002-0003 and ¶ 0005-0006]).
Chernov as modified by Jin fails to disclose circuitry to use one or more neural networks; and
determine one or more differences between the one or more second images and one or more third images that captures the 3D object from the second perspective.
However, Trenholm teaches circuitry to use one or more neural networks (Trenholm; circuitry to use one or more NNs [¶ 0064-0066]; additionally, NN operating in at least two modes [¶ 0067 and ¶ 0073]).
Chernov in view of Jin and Trenholm are considered to be analogous art because they pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, to incorporate circuitry to use one or more neural networks (as taught by Trenholm), in order to provide an improved imaging information while accurately and reliably identifying objects within images (Trenholm; [¶ 0002-0005]).
Chernov as modified by Jin and Trenholm fails to disclose determine one or more differences between the one or more second images and one or more third images that captures the 3D object from the second perspective.
However, Kawahara teaches to: determine one or more differences between the one or more second images and one or more third images that captures the 3D object from the second perspective (Kawahara; determine one or more differences between the one or more 2nd images and one or more 3rd images that captures the 3D object from the 2nd perspective [¶ 0054-0055 and ¶ 0058-0060]; moreover, difference detection [¶ 0045]; additionally, 3D object data [¶ 0042 and ¶ 0056]).
Chernov in view of Jin and Trenholm and Kawahara are considered to be analogous art because they pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, and Trenholm, to incorporate to: determine one or more differences between the one or more second images and one or more third images that captures the 3D object from the second perspective (as taught by Kawahara), in order to provide an improved modeling while reducing system resources (Kawahara; [¶ 0002-0004 and ¶ 0019]).
Regarding claim 2, Chernov in view of Jin, Trenholm, and Kawahara further discloses the one or more processors of claim 1, wherein the circuitry is to use the one or more neural networks to determine an error between the one or more images (Trenholm; using the one or more NNs to determine an error between the one or more images [¶ 0064 and ¶ 0066]; wherein, images depicting the 3D object from the multiple perspectives [¶ 0047 and ¶ 0058]; and moreover, bundle adjustment poses [¶ 0062] and pose estimation [¶ 0071-0072]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate using the one or more neural networks to determine an error between the one or more images (as taught by Trenholm), in order to provide an improved imaging information while accurately and reliably identifying objects within images (Trenholm; [¶ 0002-0005]).
Kawahara further teaches to determine an error between the one or more second images and the one or more third images depicting the 3D object from the second perspective (Kawahara; using tests to determine the one or more differences further comprise determining an implicit error (given the nature of a test) between the one or more 2nd images and the one or more 3rd images depicting the 3D object from the 2nd perspective [¶ 0058-0060]; moreover, difference detection in relation with testing and matching [¶ 0045, ¶ 0055-0056 and ¶ 0062]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate to determine an error between the one or more second images and the one or more third images depicting the 3D object from the second perspective (as taught by Kawahara), in order to provide an improved modeling while reducing system resources (Kawahara; [¶ 0002-0004 and ¶ 0019]).
Regarding claim 3, Chernov in view of Jin, Trenholm, and Kawahara further discloses the one or more processors of claim 1, wherein the circuitry is to use one or more neural networks to modify the one or more 3D mesh representations further comprise updating one or more latent vector values in latent space based, at least in part, on the one or more differences (Trenholm; the using the one or more NNs to modify the one or more 3D mesh representations [as addressed within the parent claim(s)] further comprise updating one or more latent vector values in latent space based (at least in part) on the one or more differences [¶ 0064 and ¶ 0066-0067]; wherein, a NN uses weights within layers [¶ 0079 and ¶ 0081-0082]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate the using the one or more neural networks to modify the one or more 3D mesh representations further comprise updating one or more latent vector values in latent space based, at least in part, on the one or more differences (as taught by Trenholm), in order to provide an improved imaging information while accurately and reliably identifying objects within images (Trenholm; [¶ 0002-0005]).
Kawahara further teaches updating based, at least in part, on the one or more differences (Kawahara; updating [¶ 0062-0064] based, at least in part, on the one or more differences [¶ 0055-0056 and ¶ 0058-0059]; wherein, differences are detected [¶ 0045 and ¶ 0060] in relation with changing or updating [¶ 0065-0066]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate updating based, at least in part, on the one or more differences (as taught by Kawahara), in order to provide an improved modeling while reducing system resources (Kawahara; [¶ 0002-0004 and ¶ 0019]).
Regarding claim 4, Chernov in view of Jin, Trenholm, and Kawahara further discloses the one or more processors of claim 1, wherein the circuitry is to use the one or more neural networks to modify the one or more 3D mesh representations further comprise obtaining one or more geometric constraints corresponding to a 3D object (Trenholm; the one or more NNs to modify the one or more 3D mesh representations [as addressed within the parent claim(s)] further comprise obtaining one or more geometric constraints corresponding to a 3D object [¶ 0051-0054]; additionally, segmentation algorithm [¶ 0068-0070]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate using the one or more neural networks to modify the one or more 3D mesh representations further comprise obtaining one or more geometric constraints corresponding to a 3D object (as taught by Trenholm), in order to provide an improved imaging information while accurately and reliably identifying objects within images (Trenholm; [¶ 0002-0005]).
Regarding claim 5, Chernov in view of Jin, Trenholm, and Kawahara further discloses the one or more processors of claim 1, wherein the one or more 3D mesh representations are generated by at least selecting an initial latent value (Trenholm; the one or more 3D mesh representations are generated by at least selecting an initial latent value [¶ 0064, ¶ 0066-0067, and ¶ 0076]; moreover, CNN machine learning models [¶ 0081-0082]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate the one or more 3D mesh representations are generated by at least selecting an initial latent value (as taught by Trenholm), in order to provide an improved imaging information while accurately and reliably identifying objects within images (Trenholm; [¶ 0002-0005]).
Regarding claim 7, Chernov in view of Jin, Trenholm, and Kawahara further discloses the one or more processors of claim 1, wherein resolution of the one or more 3D mesh representations is increased as a result of modification of the one or more 3D mesh representations (Chernov; resolution of the one or more 3D mesh representations is increased as a result of modification (i.e. up-sampling) of the one or more 3D mesh representations [¶ 0109-0111 and ¶ 0116]; additionally, estimated ambiguity of pixels [¶ 0114]).
Regarding claim 8, Chernov in view of Jin, Trenholm, and Kawahara further discloses the one or more processors of claim 1, wherein the region corresponds to a meshlet that is combined with another meshlet to modify the one or more 3D mesh representations (Chernov; the region corresponds to a meshlet that is combined with another meshlet to modify the one or more 3D mesh representations [¶ 0058 and ¶ 0068-0069]; moreover, a number of patches [¶ 0093 and ¶ 0128]).
Regarding claim 9, the rejection of claim 9 is addressed within the rejection of claim 1, due to the similarities claim 9 and claim 1 share, therefore refer to the rejection of claim 1 regarding the rejection of claim 9.
Regarding claim 10, Chernov in view of Jin, Trenholm, and Kawahara further discloses the method of claim 9, further comprising extracting a value representing one or more features of the 3D object and using the value to generate the one or more 3D mesh representations (Chernov; extracting a value representing one or more features of the 3D object and using the value to cause the one or more processes to generate the one or more 3D mesh representations [¶ 0079-0080 and ¶ 0084]; moreover, aligning feature point(s) [¶ 0087], creating a feature vector [¶ 0091], and feature tracking [¶ 0093]).
Jin further teaches feature processing (Jin; constraints representing a 3D model associated with indicating features [¶ 0025-0027]; moreover, identifying features [¶ 0043-0044]; moreover, analyzing individual image data to determine object feature [¶ 0049-0051]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate feature processing (as taught by Jin), in order to provide an improved modeling while reducing system resources (Jin; [¶ 0002-0003 and ¶ 0005-0006]).
Trenholm further teaches extracting a value representing one or more features of the 3D object and using the value to cause the one or more neural networks to generate the one or more 3D mesh representations (Trenholm; extracting a value representing one or more features of the 3D object and using the value to cause the one or more neural networks to generate the one or more 3D mesh representations [¶ 0044-0045] and ¶ 0052-0054; wherein features are detected/extracted [¶ 0061]; moreover, detecting a plurality of features of the object [¶ 0022]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate extracting a value representing one or more features of the 3D object and using the value to cause the one or more neural networks to generate the one or more 3D mesh representations (as taught by Trenholm), in order to provide an improved imaging information while accurately and reliably identifying objects within images (Trenholm; [¶ 0002-0005]).
Regarding claim 11, the rejection of claim 11 is addressed within the rejection of claim 2, due to the similarities claim 11 and claim 2 share, therefore refer to the rejection of claim 2 regarding the rejection of claim 11.
Regarding claim 12, the rejection of claim 12 is addressed within the rejection of claim 8, due to the similarities claim 12 and claim 8 share, therefore refer to the rejection of claim 8 regarding the rejection of claim 12. Although, claim 12 and claim 8 may not be identical, they are considerably comparable or substantially equivalent given their overlapping subject matter. Thus, it is reasonable to reject claim 12 based on the teachings and rational in relation with the prior art within the rejection of claim 8.
Regarding claim 13, the rejection of claim 13 is addressed within the rejection of claim 8, due to the similarities claim 13 and claim 8 share, therefore refer to the rejection of claim 8 regarding the rejection of claim 13. Although, claim 13 and claim 8 may not be identical, they are considerably comparable or substantially equivalent given their overlapping subject matter. Thus, it is reasonable to reject claim 13 based on the teachings and rational in relation with the prior art within the rejection of claim 8.
Regarding claim 14, the rejection of claim 14 is addressed within the rejection of claim 7, due to the similarities claim 14 and claim 7 share, therefore refer to the rejection of claim 7 regarding the rejection of claim 14. Although, claim 14 and claim 7 may not be identical, they are considerably comparable or substantially equivalent given their overlapping subject matter. Thus, it is reasonable to reject claim 14 based on the teachings and rational in relation with the prior art within the rejection of claim 7.
Regarding claim 15, the rejection of claim 15 is addressed within the rejection of claim 1, due to the similarities claim 15 and claim 1 share, therefore refer to the rejection of claim 1 regarding the rejection of claim 15.
Chernov teaches a non-transitory computer readable medium storing instructions that, when executed by a processor cause the processor to perform operations (Chernov; a non-transitory computer readable medium storing instructions that, when executed by a processor perform a method/operations [¶ 0012 and ¶ 0189]).
(further refer to the rejection of claim 1)
Regarding claim 17, Chernov in view of Jin, Trenholm, and Kawahara further discloses the non-transitory computer readable medium of claim 15, wherein using the one or more processes to modify the one or more 3D mesh representations further comprises increasing a resolution of the one or more 3D mesh representations using one or more geometric constraints imposed on the one or more objects (Chernov; using the one or more processes to modify the one or more 3D mesh representations [as addressed within the parent claim(s)] further comprises increasing a resolution of the one or more 3D mesh representations [¶ 0109-011 and ¶ 0116] using one or more implicit geometric constraints imposed on the one or more objects [¶ 0058 and ¶ 0068-0070]), and wherein the one or more geometric constraints correspond to one or more conditions of a set of conditions under which the one or more first images were captured (Chernov; the one or more implicit geometric constraints correspond to one or more conditions/parameters of a set of conditions under which the one or more 1st images were captured [¶ 0061-0062 an ¶ 0064-0066]).
Trenholm further teaches wherein using the one or more neural networks to modify the one or more 3D mesh representations further comprises increasing a resolution of the one or more 3D mesh representations using one or more geometric constraints imposed on the one or more objects (Trenholm; using the one or more NNs [as addressed within the parent claim(s)] to modify the one or more 3D mesh representations further comprises increasing an implicit resolution (given dynamic range) of the one or more 3D mesh representations using one or more geometric constraints imposed on the one or more objects [¶ 0053-0054 and ¶ 0059]; moreover, segmentation [¶ 0068-0070]), and wherein the one or more geometric constraints correspond to one or more conditions of a set of conditions under which the one or more first images were captured (Trenholm; the one or more geometric constraints correspond to one or more conditions of a set of conditions under which the one or more 1st images were captured [¶ 0058-0059]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate using the one or more neural networks to modify the one or more 3D mesh representations further comprises increasing a resolution of the one or more 3D mesh representations using one or more geometric constraints imposed on the one or more objects, and wherein the one or more geometric constraints correspond to one or more conditions of a set of conditions under which the one or more first images were captured (as taught by Trenholm), in order to provide an improved imaging information while accurately and reliably identifying objects within images (Trenholm; [¶ 0002-0005]).
Regarding claim 18, the rejection of claim 18 is addressed within the rejection of claim 8, due to the similarities claim 18 and claim 8 share, therefore refer to the rejection of claim 8 regarding the rejection of claim 18. Although, claim 18 and claim 8 may not be identical, they are considerably comparable or substantially equivalent given their overlapping subject matter. Thus, it is reasonable to reject claim 18 based on the teachings and rational in relation with the prior art within the rejection of claim 8.
Regarding claim 19, the rejection of claim 19 is addressed within the rejection of claim 2, due to the similarities claim 19 and claim 2 share, therefore refer to the rejection of claim 2 regarding the rejection of claim 19.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chernov in view of Jin, Trenholm, and Kawahara as applied to claim(s) 1 above, and further in view of Quaroni et al., US Patent No. 8669980 B1, hereinafter Quaroni.
Regarding claim 6, Chernov in view of Jin, Trenholm, and Kawahara further discloses the one or more processors of claim 1, wherein the circuitry is to modify the one or more 3D mesh representations (Chernov; the circuitry is to modify the one or more 3D mesh representations [¶ 0057-0058], as illustrated within Fig. 2A; moreover, mapping textures on to a surface mesh [¶ 0066-0069]).
Chernov in view of Jin, Trenholm, and Kawahara fails to disclose to modify the one or more 3D mesh representations to increase resolution.
However, Quaroni teaches to modify the one or more 3D mesh representations to increase resolution (Quaroni; to modify the one or more 3D mesh representations to increase resolution [Col. 9, lines 25-60]).
Chernov in view of Jin, Trenholm, and Kawahara and Quaroni are considered to be analogous art because they pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate to modify the one or more 3D mesh representations to increase resolution (as taught by Quaroni), in order to provide improved computer generated imaging that is appealing using high levels of detail (Quaroni; [Col. 2, lines 4-41]).
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chernov in view of Jin, Trenholm, and Kawahara as applied to claim(s) 15 above, and further in view of Florez Choque, US Patent No. 10311334 B1, hereinafter Florez-Choque.
Regarding claim 16, Chernov in view of Jin, Trenholm, and Kawahara further discloses the non-transitory computer readable medium of claim 15, wherein the instructions, when executed by the processor, further cause the processor (Chernov; the instructions, when executed by the processor, further cause the processor [¶ 0189]) to at least:
to generate the one or more 3D mesh representations of the 3D object (Chernov; the processor [as addressed above] is configured to use a processing stage to generate the one or more 3D mesh representations of the 3D object [¶ 0058 and ¶ 0068-0070]).
Chernov as modified by Jin, Trenholm, and Kawahara fails to disclose use a decoder to generate the one or more representations of the object, wherein the decoder receives one or more latent vector values as input and generates a decoded representation of the one or more latent vector values.
However, Florez-Choque teaches use a decoder to generate the one or more representations of the object (Florez-Choque; use an implicit decoder (given VAE) to generate the one or more representations of the object [Col. 7, lines 12-60 and Col. 8, lines 1-35]; wherein, the VAE comprises an encoder/compression and decoder/reconstruction Col. 4, line 66 to Col. 5, line 35), wherein the decoder receives one or more latent vector values as input and generates a decoded representation of the one or more latent vector values (Florez-Choque; the implicit decoder (given VAE, comprises a reconstruction) [as addressed above] receives one or more latent vector values [Col. 3, lines 28-45 and Col. 4, line 66 to Col. 5, line 35] as input and generates a decoded/reconstructed representation of the one or more latent vector values [Col. 7, lines 12-60 and Col. 8, lines 1-35]).
Chernov in view of Jin, Trenholm, and Kawahara and Florez-Choque are considered to be analogous art because they pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate one or more neural networks comprise a variational autoencoder (VAE) (as taught by Florez-Choque), in order to provide an improved recognition and reconstruction of image data that utilities an optimized neural network (Florez-Choque; [Col. 1, lines 7-50]).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chernov in view of Jin, Trenholm, and Kawahara as applied to claim(s) 19 above, and further in view of O’Keefe et al., US PGPUB No. 20150332464 A1, hereinafter O’Keefe.
Regarding claim 20, Chernov in view of Jin, Trenholm, and Kawahara further discloses the non-transitory computer readable medium of claim 19, wherein the one or more second images comprise error (Chernov; [¶ 0101 and ¶ 0106]; wherein, refinement of camera positions is performed to decrease measured noise [id.]).
Chernov in view of Jin, Trenholm, and Kawahara fails to disclose one or more warped images.
However, O’Keefe teaches one or more second images comprise one or more warped images (O’Keefe; the one or more second images (i.e. current pose of a capture system) comprise one or more warped images [¶ 0028-0029]; moreover, derived data techniques [¶ 0024-0025]; and moreover, mismatch/error between poses [¶ 0014]).
Chernov in view of Jin, Trenholm, and Kawahara and O’Keefe are considered to be analogous art because they pertain to generating and/or managing data in relation with providing media data to a user, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Chernov as modified by Jin, Trenholm, and Kawahara, to incorporate one or more second images comprise one or more warped images (as taught by O’Keefe), in order to provide an improved imaging using automatic or semi-automatic calibration techniques associated viewpoints (O’Keefe; [¶ 0002-0003, ¶ 0010, and ¶ 0044-0045]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Balter et al. (US PGPUB No. 20070064099 A1) : regarding 3D modeling; and
Sheffield et al. (US PGPUB No. 20190362551 A1) : regarding mesh generation.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of Reference Cited for a listing of analogous art.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX ONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Charles Lloyd Beard whose telephone number is (571)272-5735. The examiner can normally be reached Monday - Friday, 8:00 AM - 5: 00 PM, alternate Fridays EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached at (571) 272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CHARLES LLOYD. BEARD
Primary Examiner
Art Unit 2611
/CHARLES L BEARD/ Primary Examiner, Art Unit 2611
Prosecution Insights