DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
There are numerous drafting issues that make the claims indefinite and unclear.
Claim 1 recites “the assets,” “the parts,” “the respective asset,” “the respective part,” “the shape variations” as bolded below, and their antecedent base are unclear.
Claim 1. A method comprising:
automatically segmenting at least some assets into respective parts, the assets comprising images of objects;
calculating how the parts in the respective asset are hierarchically placed with respect other parts in the respective asset;
associating each part with a respective numerical descriptor that describes the respective part;
calculating a similarity metric for each part that describes the respective part's similarity to other parts;
generating plural shape variations of at least one input asset using at least a first one of the parts with respective numerical descriptor and similarity matrix;
ranking at least some of the shape variations; and
outputting on at least one display images of the shape variations consistent the with ranking.
Further, Claim 1 recites “consistent the with ranking,” and the claim probably should read “consistent with the ranking.”
Significantly, Claim 1 recites “the parts in the respective asset are hierarchically placed with respect other parts in the respective asset.” The term “other parts” is unclear here. Does “other parts” overlap with “the parts” in the respective asset? If there is no overlap, is “other parts” an empty set? It adds further confusion, as the antecedent base for “the parts” and “the respective asset” are unclear.
More significantly, Claim 1 also recites “the respective part's similarity to other parts.” It appears the “other parts” could be identical to the previous recited “other parts in the respective asset.” However, such scope is inconsistent with the specification, which states:
[0044] Proceeding to state 202, a shape analysis model is executed on the segments of the asset to learn shape structure by calculating how the parts in the 3D asset are hierarchically placed with respect other parts in the asset model. Each part identified at state 202 may be associated with a numerical descriptor that describes the part. Similarity metrics also are calculated for each part that describe the part's similarity to other parts. Thus, state 202 computes shape descriptors and similarity metrics of a given part with respect to all parts in the database. . . .
Spec. ¶ 44.
The Examiner recommends the following amendments to address abovementioned indefinite issues and to clarify the claim language.
Claim 1. A method comprising:
automatically segmenting at least one asset into respective parts based on a machine learning (ML) pipeline, each of the at least one asset comprising a 3D image of an object;
calculating how each part of the respective parts in the at least one asset is to other parts of the respective parts in the at least one asset;
associating the each part with a respective numerical descriptor that describes the each
calculating a similarity metric for the each part that describes the each in a database;
generating plural shape variations of at least one input asset based on the machine learning (ML) pipeline by using at least respective parts with respective numerical descriptor and similarity matrix;
ranking generated shape variations; and
outputting on at least one display images of the generated shape variations based on the ranking.
Claim 9 recites “the assets,” “the respective assets,” “the part,” “the respective asset,” “the respective segmented part,” “the respective part,” “the adjacency graphs,” and “the shape variations” as bolded below, and their antecedent base are unclear.
Claim 9. A processor system configured to:
execute a software-implemented parts segmenter on at least some assets in a database, the assets comprising respective three dimensional (3D) images of objects, for producing segmented parts of the respective assets;
for at least some of the segmented parts, generate a respective adjacency graph, at least one shape descriptor, and a similarity matrix, the adjacency graph representing a hierarchy of the part in the respective asset, the shape descriptor quantifying shape structure of the respective segmented part, the similarity matrix representing a similarity of the respective part to other parts;
receive a command to generate plural shape variations of an input;
using the input, generate the plural shape variations at least in part using some of the adjacency graphs, shape descriptors, and similarity matrices of respective parts;
rank at least some of the plural shape variations to establish a ranking; and
present on at least one display at least some of the shape variations according to the ranking.
Claim 9 also recites “other parts” as Claim 1 does, and Claim 9 has the same deficiency.
The Examiner recommends the following amendments to address abovementioned indefinite issues and to clarify the claim language.
Claim 9. A processor system comprising at least one processor configured to:
execute a software-implemented parts segmenter on at least one asset in a database based on a machine learning (ML) pipeline, each of the at least one asset a three dimensional (3D) image of an object at least one asset
for at least some of the segmented parts, generate a at least some of the segmented parts of the at least one asset, the at least one shape descriptor quantifying shape structure of the at least some of the segmented parts a respective part of the at least some of the segmented parts to other parts of a database;
receive a command to generate plural shape variations of an input;
using the input, generate the plural shape variations by using the machine learning (ML) pipeline based on the generated adjacency graphthe at least one shape descriptorthe similarity matrix
rank at least some of the plural shape variations to establish a ranking; and
present on at least one display the at least some of the plural shape variations according to the ranking.
Claim 18 recites “an offline stage” twice and recites “the offline stage” subsequently. Therefore, the antecedent basis is unclear. Claim 18 also recites “the online stage” and “the shape variations” and their antecedent base are unclear.
Claim 18. A device comprising:
at least one computer memory that is not a transitory signal and that includes instructions executable by at least one processor system to:
execute a machine learning (ML) pipeline on 3D assets to produce shape variations of the assets without any meta data including part labels, part structure, and part similarity metrics given explicitly alongside the 3D assets, the pipeline being executed in an offline stage and an offline stage,
the offline stage comprising:
for at least some of the 3D assets, identifying individual parts of the respective 3D asset;
the online stage comprising:
receiving an input and based on the input, generating plural shape variations;
ranking at least some of the shape variations using part similarity metrics; and
presenting at least some of the shape variations on a display consistent with the ranking.
The Examiner recommends the following amendments to address abovementioned indefinite issues and to clarify the claim language.
Claim 18. A device comprising: at least one computer memory that is not a transitory signal and that includes instructions executable by at least one processor system to:
execute a machine learning (ML) pipeline on 3D assets to produce shape variations of the 3D assets without any meta data including part labels, part structure, and part similarity metrics given explicitly alongside the 3D assets, the pipeline being executed in an offline stage and an online stage,
the offline stage comprising: for at least one of the 3D assets, identifying individual parts of the at least one of the 3D assets;
the online stage comprising:
receiving an input and based on the input, generating plural shape variations;
ranking at least some of the plural shape variations using part similarity metrics for the identified individual parts; and
presenting the at least some of the plural shape variations on a display based on the ranking.
Claims 2-8, 10-17, and 19-20 are rejected because of their dependence on independent claims 1, 9, or 18 and inherit their deficiencies. Please review the dependent claims carefully for similarly issues and the Examiner recommends revision to the dependent claims to be consistent with the Examiner’s recommendations for the independent claims.
Claim Interpretation
Claim 9 recites “A processor system,” and the Examiner is reading the limitation as: a system comprising at least one processor.
Allowable Subject Matter
Claims 1-17 would be allowable if rewritten to address the 112(b) rejections substantially based on the Examiner’s recommendations and do not substantially change the scope of the claims.
The closest prior art combination Fu et al. (“ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model”) in view of Mittal et al. (“AutoSDF: Shape Priors for 3D Completion, Reconstruction, and Generation”), Chang et al. (US 20150052130 A1), and Hoxha et al. (“Retrieving images with generated textual descriptions”) does not teach:
Claim 1’s
automatically segmenting at least one asset into respective parts based on a machine learning (ML) pipeline, each of the at least one asset comprising a 3D image of an object;
calculating how each part of the respective parts in the at least one asset is to other parts of the respective parts in the at least one asset; …
calculating a similarity metric for the each part that describes the each in a database;
Claim 9’s
for at least some of the segmented parts, generate a at least some of the segmented parts of the at least one asset, the at least one shape descriptor quantifying shape structure of the at least some of the segmented parts a respective part of the at least some of the segmented parts to other parts of a database;
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Fu et al. (“ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model”) in view of Mittal et al. (“AutoSDF: Shape Priors for 3D Completion, Reconstruction, and Generation”) and Chang et al. (US 20150052130 A1).
Regarding Claim 18, Fu teaches A device comprising:
at least one computer memory that is not a transitory signal and that includes instructions executable by at least one processor system to (“We present ShapeCrafter, a neural network for recursive text-conditioned 3D shape generation. . . . Our method supports shape editing, extrapolation, and can enable new applications in human–machine collaboration for creative design.” Fu Abstract. The neural network and the human-machine collaboration is implemented by software residing on a computer comprising memory and a processor. The Examiner’s secondary reference would provide more explicitly teaching.):
execute a machine learning (ML) pipeline on 3D assets to produce shape variations of the assets
Fu:
PNG
media_image1.png
530
560
media_image1.png
Greyscale
, where shape variations of chairs, mapped to assets, are produced and shown at top of the screenshot.
Fu Fig. 6 provides clearer examples of these shape variations of a chair:
PNG
media_image2.png
114
308
media_image2.png
Greyscale
.
The machine learning (ML) pipeline is mapped to the sequence of machine learning models as shown being used in fig. 4 and as shown being trained in Fu section 4.4 Training.
PNG
media_image3.png
150
924
media_image3.png
Greyscale
Fig. 4:
PNG
media_image4.png
362
926
media_image4.png
Greyscale
),
the pipeline being executed in an offline stage and an offline stage (
[BRI on the record]
The claim has been rejected under 112(b) because “offline stage” has been recited twice and the claim later recites “the offline stage.” For the purpose of art rejection, the Examiner is treating the limitation as “in an offline stage and an online stage.”
[Mapping Analysis]
The offline stage is mapped to the training stage of the pipeline of learning models as shown in Fu section 4.4 Training, because the training is to prepare the pipeline.
The online stage is mapped to the use of the trained pipeline of learning models to infer as shown in Fu Fig. 4.),
online stage comprising:
receiving an input and based on the input,
generating plural shape variations (
Fig. 4 shows the input could be “chair slopes slightly backwards” and/or “with a rectangular cusion” at the online “inference” stage. Fig. 4 also shows a plural shape variations for the chair based on the corresponding text inputs.);
ranking at least some of the shape variations using part similarity metrics (
Fu discloses ranking based on similarity scores for each phrase sequence, mapped to part similarity metrics, stating “To get shapes corresponding to phrase sequences, we calculate the similarity score of all the phrase sequences in our dataset with RoBERTa [29]. We consider two phrase sequences as similar if their similarity score is higher than a certain threshold (we empirically chose 0.94). For each phrase sequence, its corresponding shapes include the original Text2Shape dataset pair as well as shapes paired with its similar phrase sequences. Therefore, Text2Shape++ is a many-to-many text–shape correspondence dataset.” 4.1 Text2Shape++.
The following Fu Fig. 2 shows that each phrase sequence corresponds to shape variations
PNG
media_image5.png
280
958
media_image5.png
Greyscale
); and
presenting at least some of the shape variations on a display consistent with the ranking (
Fu Fig. 2:
PNG
media_image6.png
192
498
media_image6.png
Greyscale
Selected Shape variations that match phrase(s) (text) are displayed, and the matching is based on similarity scores that determines a ranking based on the similarity scores.).
However, Fu does not explicitly disclose:
at least one computer memory that is not a transitory signal and that includes instructions executable by at least one processor system to . . . ;
execute a machine learning (ML) pipeline/model without any meta data including part labels, part structure, and part similarity metrics given explicitly alongside the 3D assets; and
the offline stage comprising: for at least some of the 3D assets, identifying individual parts of the respective 3D asset.
Mittal teaches the offline stage (fig. 2’s models including the autoregressive model is trained/”learned”) comprising: for at least some of the 3D assets (Mittal fig. 2 Input shape X), identifying individual parts (
PNG
media_image7.png
124
140
media_image7.png
Greyscale
) of the respective 3D asset (
PNG
media_image8.png
374
772
media_image8.png
Greyscale
PNG
media_image9.png
336
372
media_image9.png
Greyscale
).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Mittal’s techniques of identifying individual parts of 3D assets with Fu’s technique of generating shape variations. One of ordinary skill in the art would be motivated to achieve better results and provide flexibility to the solutions. Mittal recites, “Powerful priors allow us to perform inference with insufficient information. In this paper, we propose an autoregressive prior for 3D shapes to solve multimodal 3D tasks such as shape completion, reconstruction, and generation. We model the distribution over 3D shapes as a nonsequential autoregressive distribution over a discretized, low-dimensional, symbolic grid-like latent representation of 3D shapes. This enables us to represent distributions over 3D shapes conditioned on information from an arbitrary set of spatially anchored query locations and thus perform*indicates equal contribution shape completion in such arbitrary settings (e.g. generating a complete chair given only a view of the back leg). . . . We validate the effectiveness of the proposed method using both quantitative and qualitative evaluation and show that the proposed method outperforms the specialized stateof-the-art methods trained for individual tasks.” Mittal Abstract.
Further, Fu discloses that it uses Mittal’s method, stating:
PNG
media_image10.png
228
536
media_image10.png
Greyscale
, where “[33]” refers to Mittal.
However, Fu in view of Mittal does not explicitly disclose
at least one computer memory that is not a transitory signal and that includes instructions executable by at least one processor system to . . . ; and
execute a machine learning (ML) pipeline/model without any meta data including part labels, part structure, and part similarity metrics given explicitly alongside the 3D assets.
Chang teaches at least one computer memory that is not a transitory signal and that includes instructions executable by at least one processor system to . . . (Chang fig. 5; ¶ 46); and
execute a machine learning (ML) pipeline/model without any meta data including part labels, part structure, and part similarity metrics given explicitly alongside the 3D assets (“Unstructured information, without any metadata suggesting a category for classification, may be classified with statistical methods, machine learning methods, or ontological methods. An ontology defines the terms used to describe and represent an area of knowledge.” Chang ¶ 4.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chang’s teaching on computer and unstructured data with Fu in view of Mittal. One of ordinary skill in the art would be motivated to a) automate a process with computer(s) to provide convenience to a user, and b) allow flexibility and convenience to the user when the data are unstructured.
Regarding Claim 19, Fu in view of Mittal and Change teaches The device of claim 18, wherein the input comprises a text description of a 3D object (Fu Fig. 4 shows the input could be “chair slopes slightly backwards” and/or “with a rectangular cushion.”).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Fu in view of Mittal Chang as applied to Claim 18, in further view of Hoxha et al. (“Retrieving images with generated textual descriptions”).
Regarding Claim 20, Fu in view of Mittal and Change teaches The device of claim 18.
Fu in view of Mittal and Change does not explicitly disclose wherein the input comprises in image of a 3D object.
Hoxha teaches wherein the input comprises in image of a 3D object (
Hoxha teaches generating textual description of an input of an image of an object, stating “The first one generates textual descriptions of the content of the RS images combining a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract the features of the images and to generate the descriptions of their content, respectively.” Hoxha Abstract.
Fig. 1:
PNG
media_image11.png
122
408
media_image11.png
Greyscale
, where the input/query image contains real objects in a real environment. Such real objects are 3D.
After Fu in view of Mittal and Change is combined with Hoxha, an input image is converted to text input that describes the input image, wherein the text input is further employed by Fu in view of Mittal and Change’s method, the input of which is text.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Hoxha’s image to text conversion with Fu in view of Mittal and Chang. One of ordinary skill in the art would be motivated to provide more convenience to a user, because there are scenarios when image input are more convenient. For example, a user’s language ability is imitated, an image input provides more convenience to such users.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wang et al. (“CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields”). Note the shape editing as shown in fig. 3, and shape editing is to provide shape variations as claimed.
PNG
media_image12.png
428
452
media_image12.png
Greyscale
Khan et al. (“A generative design technique for exploring shape variations”).
Park et al. (“Generative 3D appearance design- A survey of generation, segmentation and editing by artificial intelligence”).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHENGXI LIU whose telephone number is (571)270-7509. The examiner can normally be reached M-F 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHENGXI LIU/ Primary Examiner, Art Unit 2611