DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Claims 1-20 are presented for examination.
Claim Rejections - 35 USC § 112
3. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
3.1 Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The claims provide for generating an image vector; the term image vector is vague and indefinite, as it is unclear how said image vector could be generated as claims nor what is used as input to generate said image vector. In particular, the claims do not what training data is used in the trained machines which would allow for "image vector" to be generated using the trained machine. Since the use of generated image vectors seems to be at the core of the claimed invention, this step is not sufficiently clear to define the claimed invention and its scope. It is claimed in the form of a result to be achieved, not in the form of clear steps achieving this result.
Furthermore, the claims recite a step of generating 3D point cloud model…, it is unclear to the examiner how said 3D point cloud model can be generated based on a single "image vector" using a second "trained machine learning algorithm", as this would require a previous step of training the second ML algorithm, based on training data comprising pairs of image vectors and point cloud models, which are however not claimed. Also, this step seems to be at the core of the invention. It is claimed in the form of a result to be achieved, not in the form of clear steps achieving this result.
Similarly, the step of "generating a three-dimensional CAD model” using the three-dimensional point cloud model of the object" does not have a clearly defined scope. It could comprise inter alia: a mesh model, a finite element model, a CSG model, etc… so it is not clear what and how is carried out during the generation, and what the scope of protection is.
Dependent claims 3-4 depend there from and recite second and thirst trained machines inherit the same defect.
Each of claim 1, 5, and 13 further recite acronyms “VGG” not defined by the claims and thus renders the claims indefinite; and since the term could have multiple meanings, the Examiner respectfully requests that applicant provides a clear definition for the term in response to this office correspondence.
Claim Rejections - 35 USC § 101
4. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
4.1 Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 2A- Prong One
The claim(s) recite(s) a system and method of providing a three-dimensional computer-aided design (CAD) model of an object in a CAD environment, the method comprising: The step of: “generating an image vector from the two-dimensional image using a first trained machine learning algorithm, wherein generating the image vector comprises pre-processing the two-dimensional image to generate a three-dimensional image matrix and transforming the three-dimensional image matrix into a high-dimensional image vector using a trained VGG convolutional neural network”; “generating three-dimensional points for each two-dimensional point in the generated image vector using a second trained machine learning algorithm”; “generating a three-dimensional point cloud model of the physical object based on the generated three-dimensional points”; “generating a three-dimensional CAD model of the physical object using the three-dimensional point cloud model of the physical object”, under the broadest reasonable interpretation fall under a mental process or otherwise a mathematical concept / mathematical relationship. Likewise, the further steps of: “performing a search for the three-dimensional CAD model of the object in a geometric model database comprising a plurality of three-dimensional CAD models based on the generated image vector; determining whether the requested three-dimensional CAD model of the object is successfully found in the geometric model database” in claims 5 and 13 also fall under a mental process. Therefore, the claims are directed to an abstract idea, by use of generic computer components and thus are clearly directed to an abstract idea, as constructed.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional limitation such as: “a data processing system”, “a processing unit”, “a memory”, “a CAD module”, either alone or in combination, all serve to gather and process data and do not add anything more significantly to the judicial exception, but are mere instructions to apply the exception using a generic computer component that are well known, routine, and conventional activities (see specification at para [0036]-[0042], and fig.1) which can be of any type, including general-purpose computer (para [0037]) previously known in the industries. Merely adding a programmable computer to perform generic computer functions does not automatically overcome an eligibility rejection. Alice, 573 U.S. at 223-24. Furthermore, the use of a general-purpose computer to apply an otherwise ineligible algorithm does not qualify as a particular machine. See Ultramerciallnc. v. Hulu, LLC, 772F.3d 709, 716-17 (Fed. Cir. 20l4); In re TLI Commc 'ns LLC v. AV Automotive, LLC, 823 F.3d 607, 613 (Fed. Cir. 2016) (mere recitation of concrete or tangible components is not an inventive concept); Eon Corp. IP Holdings LLC v. AT&T Mobility LLC, 785; the step of: “receiving, by a data processing system, a request for a three-dimensional CAD model of a physical object, wherein the request comprises a two-dimensional image of the physical object”, under the broadest reasonable interpretation, reasonable fall under data gathering and processing activities that are pre-solution activities, and the further step of: ”outputting the three-dimensional CAD model of the physical object on a graphical user interface” further amount to post-solution activities and are also well-known, routine and conventional activities to output the model on a GUI and are not sufficient to amount to significantly more than the judicial exception (See further MPEP 2106.05(d)(i-iv)-f); thus are not patent eligible under 35 USC 101.
Step 2B
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as previously discussed above with reference to the integration of abstract idea into a practical application, the additional elements of: “a data processing system”, “a processing unit”, “a memory”, “a CAD module”, either alone or in combination, all serve to gather and process data and do not add anything more significantly to the judicial exception, but are mere instructions to apply the exception using a generic computer component that are well known, routine, and conventional activities (see specification at para [0036]-[0042], and fig.1) which can be of any type, including general-purpose computer (para [0037]) previously known in the industries. Merely adding a programmable computer to perform generic computer functions does not automatically overcome an eligibility rejection. Alice, 573 U.S. at 223-24. Furthermore, the use of a general-purpose computer to apply an otherwise ineligible algorithm does not qualify as a particular machine. See Ultramerciallnc. v. Hulu, LLC, 772F.3d 709, 716-17 (Fed. Cir. 20l4); In re TLI Commc 'ns LLC v. AV Automotive, LLC, 823 F.3d 607, 613 (Fed. Cir. 2016) (mere recitation of concrete or tangible components is not an inventive concept); Eon Corp. IP Holdings LLC v. AT&T Mobility LLC, 785; the step of: “receiving, by a data processing system, a request for a three-dimensional CAD model of a physical object, wherein the request comprises a two-dimensional image of the physical object”, under the broadest reasonable interpretation, reasonable fall under data gathering and processing activities that are pre-solution activities, and the further step of: ”outputting the three-dimensional CAD model of the physical object on a graphical user interface” further amount to post-solution activities and are also well-known, routine and conventional activities to output the model on a GUI and are not sufficient to amount to significantly more than the judicial exception (See further MPEP 2106.05(d)(i-iv)-f); thus are not patent eligible under 35 USC 101. Therefore, using computer components amount to no more than mere instructions to perform the abstract, and thus are not sufficient to amount to significantly more than the recited abstract, as constructed.
4.2 Dependent claims 2-4, 6-12, 14-20 merely include limitations pertaining to further mathematical computations (claim 2), “storing the three-dimensional point cloud model and the generated image vector of the two-dimensional image of the physical object in a geometric model database” (WURC post-solution activities); (claim 3); “receiving a request for a three-dimensional CAD model of the object, wherein the request comprises a two-dimensional image of the object” (WURC data gathering activities); “generating an image vector from the two-dimensional image using the first trained machine learning algorithm”; “performing a search for the three-dimensional CAD model of the object in a geometric model database comprising a plurality of three-dimensional CAD models based on the generated image vector”; “determining whether the three-dimensional CAD model of the object is successfully found in the geometric model database” (mental process); and “outputting the three-dimensional CAD model of the object on a graphical user interface.” (WURC post-solution activities); (claim 4); “wherein performing the search for the three-dimensional CAD model of the object in the geometric model database using a third trained machine learning algorithm comprises: comparing the generated image vector of the two-dimensional image with each image vector associated with respective three-dimensional CAD models of the plurality of three-dimensional CAD models in the geometric model database using the third machine learning algorithm; and identifying the three-dimensional CAD model from the geometric model database based on a best match between the generated image vector and the image vector of the three-dimensional CAD model” (mental process); (claims 6 and 14); “generating a three-dimensional CAD model of the object based on the generated image vector using a second trained machine learning algorithm when the requested three-dimensional CAD model of the object is not found in the geometric model database” (mental process); and “outputting the generated three-dimensional CAD model of the object on the graphical user interface” (WURC post-solution activities); (claims 7 and 15) “wherein generating the three-dimensional CAD model of the object based on the generated image vector using the second trained machine learning model comprises: generating a three-dimensional point cloud model of the object based on the generated image vector using the second trained machine learning algorithm”; and “generating the three-dimensional CAD model of the object using the three-dimensional point cloud model of the object” (mental process); (claims 8 and 16); “storing the generated three-dimensional CAD model of the object and the generated image vector of the two-dimensional image of the object in the geometric model database” (WURC post-solution activities); (claims 9 and 17) “wherein performing the search for the three-dimensional CAD model of the object in the geometric model database comprises: performing the search for the three-dimensional CAD model of the object in the geometric model database using a third trained machine learning algorithm” (mental process); (claims 10 and 18) “wherein performing the search for the three-dimensional CAD model of the object in the geometric model database using the third trained machine learning algorithm comprises: comparing the generated image vector of the two-dimensional image with each image vector associated with respective geometric models of the plurality of three-dimensional CAD models in the geometric model database using the third machine learning algorithm; and identifying one or more three-dimensional CAD models of the plurality of three-dimensional CAD models from the geometric model database based on a match between the generated image vector and the respective image vector of the one or more three-dimensional CAD models” (mental process); (claims 11 and 19) “ranking the one or more three-dimensional CAD models based on the match with the requested three-dimensional CAD model of the object; and determining at least one three-dimensional CAD model of the one or more three-dimensional CAD models having an image vector that best matches with the generated image vector of the two-dimensional image based on the ranking of the one or more three-dimensional CAD models” (mental process); (claims 12 and 20) “modifying the at least one determined three-dimensional CAD model based on the generated image vector of the two-dimensional model” (mental process); all of which further amount to further mental process similar to that already recited by the independent claims and already addressed above and thus are further not patent eligible under 35 USC 101.
Claim Rejections - 35 USC § 102
5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
5.0 Claim(s) 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Dal Mutto et al. (USPG_PUB No. 2018/0046649).
5.1 In considering claims 1, 5, and 13, Dal Mutto et al. teaches a method of providing a three-dimensional computer-aided design (CAD) model of an object in a CAD environment, the method comprising:
receiving, by a data processing system, a request for a three-dimensional CAD model of a physical object, wherein the request comprises a two-dimensional image of the physical object (see fig.7A-B, para [0104-0106], [0106] In the case where the media document of the query includes one or more 2D images of the same object and the database is a collection of 3D models, one or more feature vectors may be extracted from the 2D images to generate feature vectors that can be compared with feature vectors of the collection of 3D models. In one embodiment of the present invention, the 2D images are used to synthesize a 3D model using, for example, stereoscopic algorithms such as block matching, which are described briefly above); generating an image vector from the two-dimensional image using a first trained machine learning algorithm, wherein generating the image vector comprises pre-processing the two-dimensional image to generate a three-dimensional image matrix (see fig.7A-8, [0104]-[0106], Referring to FIGS. 7A and 7B, in operation 710 the processor renders the 3D model from multiple angles to generate multiple two-dimensional (2D) views 712 of the 3D model. A similar feature estimation or principal component analysis technique may also be applied to identify “preferred views” of the 3D model from which to generate the multiple 2D views of the 3D model. In operation 730, the processor supplies each of the 2D views to a convolutional neural network 732 to generate a plurality of corresponding single view feature vectors (feature vectors corresponding to single ones of the 2D views). In some embodiments, as shown in FIG. 7B, there is a separate convolutional neural network for each view. In other embodiments, each of the views is supplied to the same convolutional neural network. In operation 750, a view pooling layer 752 aggregates the single view feature vectors computed from the individual 2D views or images.) and transforming the three-dimensional image matrix into a high-dimensional image vector using a trained VGG convolutional neural network (see fig.7-8, para [0135] The query of the database may be performed in substantially the same manner shown in operations 510 and 530 of FIG. 5. In operation 510, a feature vector is computed (or extracted) based on the 3D model of the query using a trained convolutional neural network (CNN). (The weights of the CNN that has been trained on such a database may be pre-computed, and the feature vectors for each of the 3D models in the database may be pre-computed using the same CNN and those feature vectors may be stored within the database.) The feature vector of the 3D model of the query can therefore be used for classification and retrieval purposes.); generating three-dimensional points for each two-dimensional point in the generated image vector using a second trained machine learning algorithm (see fig.6-8, para [0098] As noted above, in the case where the media document of the query is a 2D image and the media documents of the database are also 2D images, the 2D image may be supplied directly to a trained CNN (e.g., trained on 2D images). In the case where the media document is an audio recording or where the media document is a video recording that includes an audio component, the processor converts the audio into one or more spectrograms and supplies the one or more spectrograms as input to the trained CNN (e.g., trained on spectrograms of audio recordings). [0099] In the case where the media document of the query is a 3D model and the database stores a collection of 2D images, one or more feature vectors can be extracted from the 3D model. For example, in one embodiment, the 3D model is rendered from multiple viewpoints to generate a plurality of 2D images, and each of the 2D images may be supplied to a trained CNN to generate a plurality of feature vectors (extracted from 2D views) associated with the 3D model. As such, the 3D model can be used to query a database of 2D images. [0104] Referring to FIGS. 7A and 7B, in operation 710 the processor renders the 3D model from multiple angles to generate multiple two-dimensional (2D) views 712 of the 3D model. A similar feature estimation or principal component analysis technique may also be applied to identify “preferred views” of the 3D model from which to generate the multiple 2D views of the 3D model. In operation 730, the processor supplies each of the 2D views to a convolutional neural network 732 to generate a plurality of corresponding single view feature vectors (feature vectors corresponding to single ones of the 2D views).); generating a three-dimensional point cloud model of the physical object based on the generated three-dimensional points (see fig.6-8, para [0047], The depth image can be obtained by different methods including geometric or electronic methods. A depth image may be represented as a point cloud or may be converted into a point cloud. [0102] In operation 650, the processor generates a feature vector from the voxels generated in operation 630. According to one embodiment of the present invention, the feature vector is 654 is computed by supplying the voxels to a trained convolutional neural network 652. Because the voxel representation can be regarded as a 3D tensor, the voxels can be directly supplied as input to a CNN, where the CNN is trained based on voxelized 3D models. [0109] In one embodiment of the present invention, where the media document of the query is a 3D model and the database of entries contains a collection of 3D models, the similar 3D models are identified in the database by applying an iterative closest point (ICP) technique. Generally, iterative closest point attempts to align two point clouds (e.g., corresponding to 3D models). One byproduct of an ICP technique is a map of distances between the points of one point cloud and the points of the other point cloud. As such, in one embodiment, a similarity metric corresponds to a statistic of these distances, such as the mean distance between points of the point clouds.); generating a three-dimensional CAD model of the physical object using the three-dimensional point cloud model of the physical object (see fig6-8, para [0100] In operation 630, the processor voxelizes the model to generate a set of voxels representing the 3D model. In one embodiment, in the voxelization process, the processor divides the bounding box into subunits, referred to as voxels. [0133] In operation 810, 3D scanner acquires the three-dimensional geometry and of the texture of a particular physical three dimensional object, such as a lamp. A portion of the computation for computing the 3D geometry and texture of the captured raw data (e.g., two dimensional images) to generate a 3D model can be performed on the 3D scanner, while the remaining portion may be performed on a local or remote server. The output of this operation is a textured three-dimensional model (3D model) of the particular object (e.g., a 3D model of the lamp).); and outputting the three-dimensional CAD model of the physical object on a graphical user interface (see fig.8, para [0133] In operation 810, 3D scanner acquires the three-dimensional geometry and of the texture of a particular physical three dimensional object, such as a lamp. A portion of the computation for computing the 3D geometry and texture of the captured raw data (e.g., two dimensional images) to generate a 3D model can be performed on the 3D scanner, while the remaining portion may be performed on a local or remote server. The output of this operation is a textured three-dimensional model (3D model) of the particular object (e.g., a 3D model of the lamp) [0137] In operation 840, all of the automatically generated metadata fields for the 3D model supplied as the query have been populated by the server, and they are displayed to the user for validation. In operation 850, the user can validate and modify the automatically generated values of the metadata suggested by the server. Once the metadata is validated (and possibly edited), the resulting metadata can be output in operation 860, with the 3D model for use in other contexts, such as the creation of an e-commerce listing including a three-dimensional model, where the automatically generated metadata can be used to automatically fill various portions of the e-commerce listing). Dal Mutto et al. further the steps of: performing a search for the three-dimensional CAD model of the object in a geometric model database comprising a plurality of three-dimensional CAD models based on the generated image vector (see fig.8 (820) para [0107] In operation 530, the processor searches for media documents in the database having feature vectors similar to the feature vector computed for the input media document. [0134] In operation 820, the database of 3D models is queried using the captured 3D model. For example, according to one embodiment of the present invention, the captured 3D model (e.g., the captured model of a lamp) is then uploaded to a local or remote server, which performs a search for this 3D model with respect to a database of classified and labeled 3D models.); determining whether the three-dimensional CAD model of the object is successfully found in the geometric model database ([0136] In operation 830, metadata fields are automatically generated for the 3D model of the query. As noted above, the classification may be a set of classes for the 3D model of the query when its feature vector is supplied to a classifier, and the retrieval output may be a set of entries having 3D models that are similar to the query model (e.g., having similar feature vectors). This set of similar entry models can be used for the automatic population of the metadata fields other than the class, such as the name, tags and textual description. [0137] In operation 840, all of the automatically generated metadata fields for the 3D model supplied as the query have been populated by the server, and they are displayed to the user for validation. In operation 850, the user can validate and modify the automatically generated values of the metadata suggested by the server. Once the metadata is validated (and possibly edited), the resulting metadata can be output in operation 860,); and outputting the three-dimensional CAD model of the object on a graphical user interface when the requested three-dimensional CAD model of the object is successfully found in the geometric model database (see para [0136] In operation 830, metadata fields are automatically generated for the 3D model of the query. As noted above, the classification may be a set of classes for the 3D model of the query when its feature vector is supplied to a classifier, and the retrieval output may be a set of entries having 3D models that are similar to the query model (e.g., having similar feature vectors). This set of similar entry models can be used for the automatic population of the metadata fields other than the class, such as the name, tags and textual description. [0137] In operation 840, all of the automatically generated metadata fields for the 3D model supplied as the query have been populated by the server, and they are displayed to the user for validation. In operation 850, the user can validate and modify the automatically generated values of the metadata suggested by the server. Once the metadata is validated (and possibly edited), the resulting metadata can be output in operation 860,), of claims 5 and 13.
5.2 Regarding claims 2, 8, and 16, Dal Mutto et al. teaches the step of storing the three-dimensional point cloud model and the generated image vector of the two-dimensional image of the physical object in a geometric model database (see para [0075]-[0077}, The generated models may be stored in a standard format such as a “ply” format or “obj” format and can be displayed on a display device using viewer software. In some cases, the viewing software may be web based (e.g., executed by a web browser), such as the case with the ThreeJS viewer. [0076] Storage of Three-Dimensional Models. [0077] According to one aspect of embodiments of the present invention, a collection of existing media documents, such as three-dimensional (3D) models, is stored in a database).
5.3 As per claim 3, Dal Mutto et al. teaches the step of receiving a request for a three-dimensional CAD model of the object, wherein the request comprises a two-dimensional image of the object ((see fig.7A-B, para [0104-0106], [0106] In the case where the media document of the query includes one or more 2D images of the same object and the database is a collection of 3D models, one or more feature vectors may be extracted from the 2D images to generate feature vectors that can be compared with feature vectors of the collection of 3D models. In one embodiment of the present invention, the 2D images are used to synthesize a 3D model using, for example, stereoscopic algorithms such as block matching, which are described briefly above); generating an image vector from the two-dimensional image using the first trained machine learning algorithm (see fig.7A-8, [0104]-[0106], Referring to FIGS. 7A and 7B, in operation 710 the processor renders the 3D model from multiple angles to generate multiple two-dimensional (2D) views 712 of the 3D model. A similar feature estimation or principal component analysis technique may also be applied to identify “preferred views” of the 3D model from which to generate the multiple 2D views of the 3D model. In operation 730, the processor supplies each of the 2D views to a convolutional neural network 732 to generate a plurality of corresponding single view feature vectors (feature vectors corresponding to single ones of the 2D views). In some embodiments, as shown in FIG. 7B, there is a separate convolutional neural network for each view. In other embodiments, each of the views is supplied to the same convolutional neural network. In operation 750, a view pooling layer 752 aggregates the single view feature vectors computed from the individual 2D views or images.); performing a search for the three-dimensional CAD model of the object in a geometric model database comprising a plurality of three-dimensional CAD models based on the generated image vector (see fig.8 (820) para [0107] In operation 530, the processor searches for media documents in the database having feature vectors similar to the feature vector computed for the input media document. [0134] In operation 820, the database of 3D models is queried using the captured 3D model. For example, according to one embodiment of the present invention, the captured 3D model (e.g., the captured model of a lamp) is then uploaded to a local or remote server, which performs a search for this 3D model with respect to a database of classified and labeled 3D models.); determining whether the three-dimensional CAD model of the object is successfully found in the geometric model database ([0136] In operation 830, metadata fields are automatically generated for the 3D model of the query. As noted above, the classification may be a set of classes for the 3D model of the query when its feature vector is supplied to a classifier, and the retrieval output may be a set of entries having 3D models that are similar to the query model (e.g., having similar feature vectors). This set of similar entry models can be used for the automatic population of the metadata fields other than the class, such as the name, tags and textual description. [0137] In operation 840, all of the automatically generated metadata fields for the 3D model supplied as the query have been populated by the server, and they are displayed to the user for validation. In operation 850, the user can validate and modify the automatically generated values of the metadata suggested by the server. Once the metadata is validated (and possibly edited), the resulting metadata can be output in operation 860,); and outputting the three-dimensional CAD model of the object on a graphical user interface (see para [0136] In operation 830, metadata fields are automatically generated for the 3D model of the query. As noted above, the classification may be a set of classes for the 3D model of the query when its feature vector is supplied to a classifier, and the retrieval output may be a set of entries having 3D models that are similar to the query model (e.g., having similar feature vectors). This set of similar entry models can be used for the automatic population of the metadata fields other than the class, such as the name, tags and textual description. [0137] In operation 840, all of the automatically generated metadata fields for the 3D model supplied as the query have been populated by the server, and they are displayed to the user for validation. In operation 850, the user can validate and modify the automatically generated values of the metadata suggested by the server. Once the metadata is validated (and possibly edited), the resulting metadata can be output in operation 860,).
5.4 With regards to claims 4, 10, and 18, Dal Mutto et al. teaches that wherein performing the search for the three-dimensional CAD model of the object in the geometric model database using a third trained machine learning algorithm (see the multiple CNNs shown by fig. 7B) comprises: comparing the generated image vector of the two-dimensional image with each image vector associated with respective three-dimensional CAD models of the plurality of three-dimensional CAD models in the geometric model database using the third machine learning algorithm (see para [0106] In the case where the media document of the query includes one or more 2D images of the same object and the database is a collection of 3D models, one or more feature vectors may be extracted from the 2D images to generate feature vectors that can be compared with feature vectors of the collection of 3D models. [0108] In one embodiment of the present invention, similar media documents are identified in the database by comparing the feature vector of the media document of the query with the feature vector of every entry in the database. According to another embodiment of the present invention, similar entries are grouped together (or binned) in the database based on similarity of their feature vectors. An initial search may identify one or more bins of entries that are similar to the media document of the query, where all of the entries each of the identified bins of entries may be considered to be similar to the feature vector of the media document of the query. The search may be further refined by comparing the feature vector of the media document of the query with each feature vector of each entry in each of the identified bins. [0110], The feature vectors extracted from the 2D images of the query may then be compared (e.g., using the L.sup.1 or L.sup.2 metrics described above) with the feature vectors of the 2D views of the 3D model to calculate a similarity between the 2D images of the query and the 3D model.); and identifying the three-dimensional CAD model from the geometric model database based on a best match between the generated image vector and the image vector of the three-dimensional CAD model (see para [0108] In one embodiment of the present invention, similar media documents are identified in the database by comparing the feature vector of the media document of the query with the feature vector of every entry in the database. According to another embodiment of the present invention, similar entries are grouped together (or binned) in the database based on similarity of their feature vectors. An initial search may identify one or more bins of entries that are similar to the media document of the query, where all of the entries each of the identified bins of entries may be considered to be similar to the feature vector of the media document of the query. The search may be further refined by comparing the feature vector of the media document of the query with each feature vector of each entry in each of the identified bins.).
5.5 As per claims 6 and 14, Dal Mutto et al. teaches the step of generating a three-dimensional CAD model of the object based on the generated image vector using a second trained machine learning algorithm when the requested three-dimensional CAD model of the object is not found in the geometric model database (see fig.7A-8, para [0011], The computing the feature vector may include: computing a 3D model of an object from the one or more 2D images; and extracting the feature vector from the 3D model. [0135] The query of the database may be performed in substantially the same manner shown in operations 510 and 510 of FIG. 5. In operation 510, a feature vector is computed (or extracted) based on the 3D model of the query using a trained convolutional neural network (CNN). (The weights of the CNN that has been trained on such a database may be pre-computed, and the feature vectors for each of the 3D models in the database may be pre-computed using the same CNN and those feature vectors may be stored within the database.) The feature vector of the 3D model of the query can therefore be used for classification and retrieval purposes. [0136] In operation 830, metadata fields are automatically generated for the 3D model of the query); and outputting the generated three-dimensional CAD model of the object on the graphical user interface (see fig.8 (860), para [0133] In operation 810, 3D scanner acquires the three-dimensional geometry and of the texture of a particular physical three-dimensional object, such as a lamp. A portion of the computation for computing the 3D geometry and texture of the captured raw data (e.g., two dimensional images) to generate a 3D model can be performed on the 3D scanner, while the remaining portion may be performed on a local or remote server. The output of this operation is a textured three-dimensional model (3D model) of the particular object (e.g., a 3D model of the lamp).
5.6 Regarding claims 7 and 15, Dal Mutto et al. teaches that wherein generating the three-dimensional CAD model of the object based on the generated image vector using the second trained machine learning model comprises: generating a three-dimensional point cloud model of the object based on the generated image vector using the second trained machine learning algorithm (see the multiple CNNs used in the process of fig.7A-8, and para [0047], The depth image can be obtained by different methods including geometric or electronic methods. A depth image may be represented as a point cloud or may be converted into a point cloud. [0102] In operation 650, the processor generates a feature vector from the voxels generated in operation 630. According to one embodiment of the present invention, the feature vector is 654 is computed by supplying the voxels to a trained convolutional neural network 652. Because the voxel representation can be regarded as a 3D tensor, the voxels can be directly supplied as input to a CNN, where the CNN is trained based on voxelized 3D models. [0109] In one embodiment of the present invention, where the media document of the query is a 3D model and the database of entries contains a collection of 3D models, the similar 3D models are identified in the database by applying an iterative closest point (ICP) technique. Generally, iterative closest point attempts to align two point clouds (e.g., corresponding to 3D models). One byproduct of an ICP technique is a map of distances between the points of one point cloud and the points of the other point cloud. As such, in one embodiment, a similarity metric corresponds to a statistic of these distances, such as the mean distance between points of the point clouds.); and generating the three-dimensional CAD model of the object using the three-dimensional point cloud model of the object (see fig6-8, para [0100] In operation 630, the processor voxelizes the model to generate a set of voxels representing the 3D model. In one embodiment, in the voxelization process, the processor divides the bounding box into subunits, referred to as voxels. [0133] In operation 810, 3D scanner acquires the three-dimensional geometry and of the texture of a particular physical three dimensional object, such as a lamp. A portion of the computation for computing the 3D geometry and texture of the captured raw data (e.g., two dimensional images) to generate a 3D model can be performed on the 3D scanner, while the remaining portion may be performed on a local or remote server. The output of this operation is a textured three-dimensional model (3D model) of the particular object (e.g., a 3D model of the lamp).
5.7 As per claims 9 and 17, Dal Mutto et al. teaches that wherein performing the search for the three-dimensional CAD model of the object in the geometric model database comprises: performing the search for the three-dimensional CAD model of the object in the geometric model database using a third trained machine learning algorithm (see fig.8 (820) para [0107] In operation 530, the processor searches for media documents in the database having feature vectors similar to the feature vector computed for the input media document. [0134] In operation 820, the database of 3D models is queried using the captured 3D model. For example, according to one embodiment of the present invention, the captured 3D model (e.g., the captured model of a lamp) is then uploaded to a local or remote server, which performs a search for this 3D model with respect to a database of classified and labeled 3D models.).
5.8 With regards to claims 11 and 19, Dal Mutto et al. teaches the step of ranking the one or more three-dimensional CAD models based on the match with the requested three-dimensional CAD model of the object and determining at least one three-dimensional CAD model of the one or more three-dimensional CAD models having an image vector that best matches with the generated image vector of the two-dimensional image based on the ranking of the one or more three-dimensional CAD models (see para [0021]-[0022], The collection of media documents may include a collection of two-dimensional (2D) images, the instructions for computing the feature vector may include instructions that, when executed by the processor, cause the processor to compute one or more feature vectors of the 3D model, the one or more matching media documents may include one or more matching 2D images of the collection of 2D images, and each of the corresponding feature vectors of the matching 2D images may be similar to at least one of the one or more feature vectors of the 3D model. The system may further include a three-dimensional scanner including: two or more infrared (IR) cameras; and one or more IR collimated illuminators, wherein the memory may further store instructions that, when executed by the processor, cause the processor to capture the 3D model using the three-dimensional scanner. [0022] The media document may include one or more two-dimensional (2D) images, the collection of media documents may include a collection of three-dimensional (3D) models, the instructions for computing the feature vector may include instructions that, when executed by the processor, cause the processor to compute one or more feature vectors of the one or more 2D images, the one or more matching media documents may include one or more matching 3D models of the collection of 3D models, and each of the corresponding feature vectors of the matching 3D models may be similar to at least one of the one or more feature vectors of the 3D model.).
5.9 As per claims 12 and 20, Dal Mutto et al. teaches modifying the at least one determined three-dimensional CAD model based on the generated image vector of the two-dimensional model (see para [0097], [0136] In operation 830, metadata fields are automatically generated for the 3D model of the query. As noted above, the classification may be a set of classes for the 3D model of the query when its feature vector is supplied to a classifier, and the retrieval output may be a set of entries having 3D models that are similar to the query model (e.g., having similar feature vectors). [0137] In operation 840, all of the automatically generated metadata fields for the 3D model supplied as the query have been populated by the server, and they are displayed to the user for validation. In operation 850, the user can validate and modify the automatically generated values of the metadata suggested by the server).
Conclusion
6. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
6.1 Wang et al. (USPG_PUB No. 2009/0040225) teaches an apparatus and method for three-dimensional model retrieval.
6.2 Lee et al. (USPG_PUB No. 2016/0071318) teaches methods and systems are described for generating a three-dimensional (3D) model of an object represented in a scene.
7. Claims 1-20 are rejected and this action is non-final. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDRE PIERRE-LOUIS whose telephone number is (571)272-8636. The examiner can normally be reached M-F 9:00 AM-5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, EMERSON C PUENTE can be reached at 571-272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDRE PIERRE LOUIS/Primary Patent Examiner, Art Unit 2187 June 26, 2026
Prosecution Insights