DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Responsive to communication filed on 28 February 2023.
Claims 1-20 are presented for examination.
Drawings
The drawings dated 02/28/2023 have been reviewed. They are accepted.
Specification
The abstract dated 02/28/2023 has been reviewed. It contains 109 words 8 lines and no legal phraseology. It is accepted.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
1. Claims 1-3, 5, 9, and 10 are rejected under 35 U.S.C. 102(a)(1) based on public use or sale or other public availability of the invention. The reference in question is a video posted to YouTube titled “3D-DSPNet: Product Disassembly Sequence Planning” under the Digital Experience ATL channel on 16 Nov 2021 (Hereafter referred to as Digital_2021). Although Digital_2021 is a presentation given by an inventor of the instant application, it falls outside the one year grace period for public disclosure by inventors/applicants (the effective filling date of the instant application being 28 Feb 2023), thus making it fall into public availability and open to use as a rejection under 35 U.S.C. 102. For the purposes of examination, the examiner will either quote the presenter in the video directly (providing a timestamp) or refer to screenshots in the attached “Reference Screenshots” file.
Claim 1. Digital_2021 teaches “A method comprising (Fig. 2 displays a process for the invention): generating, by one or more processors, an assembly graph based on a 3D model of a product (Fig. 2: An assembly graph (step 2) is shown that is based on a 3D CAD model (step 1). NOTE: The presenter in the video also mentions at timestamp (TS) 1:19: “The CAD model is first passed through the assembly descriptor module to pre-process the CAD file to graph based representation”, wherein the product comprises a plurality of components (Fig. 2: The assembly descriptor shows a plurality of nodes referring to components), generating, by one or more processors, one or more component graphs (Fig. 2, further clarified by Fig. 3 with labeled “Component graphs”), wherein the one or more component graphs are generated based on one or more components of the plurality of components of the product (Fig. 3 shows the component graphs correlate to individual components in the assembly graph), executing an encoder against the assembly graph and the one or more component graphs to produce a set of graph embeddings, the encoder executable by the one or more processors (Fig. 3 shows an encoder taking an assembly graph and component graphs as input and outputting graph embeddings), executing a decoder against the set of graph embeddings (Fig. 4) to produce disassembly data, wherein the decoder is executable by the one or more processors, and wherein the disassembly data identifies an order in which to remove each component of the plurality of components during disassembly of the product (Fig. 6 (A-E) shows an example model being sequentially disassembled. The presenter also mentions at TS 1:58: “The decoder is a recurrent neural network… to generate a sequence of components along with their direction.”)
Claim 2. Digital_2021 teaches the limitations of claim 1. Digital_2021 also teaches “the disassembly data indicates a disassembly direction for each component of the plurality of components (Fig. 6 (A-E) shows that each component is disassembled along an indicated direction. For example, figure 6 (B) shows the first component is removed along the x-axis (i.e. (1, 0, 0), correlating to the x, y, and z axis respectively). The presenter in the video also states at TS 1:58: “The decoder is a recurrent neural network… to generate a sequence of components along with their direction.”).
Claim 3. Digital_2021 teaches the limitations of claim 2. Digital_2021 also teaches “the decoder is configured to determine the disassembly direction for each component of the plurality of components based, at least in part, on a principal axis of alignment for the 3D model of the product (Fig. 5 shows the training dataset includes a “principle_axis”. This is further enforced by the presenter stating at TS 2:40: “We manually annotate every CAD model in the dataset with the information about the… principal axis.” NOTE: This dataset (built on top of the DMU Net dataset (Fig. 7)) was used to train the recurrent neural network in determining disassembly sequence/directions (the presenter at TS 2:12: “We used CAD models available in DMU Net for training and testing)).
Claim 5. Digital_2021 teaches the limitations of claim 2. Digital_2021 also teaches “the decoder is configured to determine the disassembly direction for each component of the plurality of components based on a set of pre-determined directions (Fig. 5: Seen near the bottom of the Assembly Json file is the “disassembly_seq” which correlates to an indexed disassembly sequence with pre-set disassembly directions (i.e. [1, 0, 0] will correlate to the X, Y, Z directions respectively (which can be seen in Fig. 6)). Similar to claim 3, this data has been used to train the recurrent neural network in determining disassembly sequence/directions (TS 2:09).
Claim 10. Digital_2021 teaches the limitations of claim 1. Digital_2021 also teaches “the encoder is a bi-level encoder (Fig. 3 illustrates how there are multiple layers (i.e. levels) in the encoder. The presenter in the video also states at TS 1:40: “The encoder is a bi-level graph neural network…”). and wherein the set of graph embeddings are generated by encoding the one or more component graphs to produce a set of component graph embeddings (Fig. 3), generating an updated assembly graph based on the set of component graph embeddings (Fig. 3), generating the set of graph embeddings based on the updated assembly graph (Fig. 3: An updated assembly graph, seen after the component graph embeddings, is run through a graph convolutional network (GCN) to produce a set of graph embeddings as the node features of the assembly graph).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
(2) Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Digital_2021 in view of Zhu_2021 (“Spatial regression graph convolutional neural networks: A deep learning paradigm for spatial multivariate distributions”, 2021).
Claim 4. Digital_2021 teaches the limitations of claim 2. Digital_2021 also makes obvious “the decoder is configured to determine the disassembly direction for each component of the plurality of components (Fig. 6 (A-E) shows that each component is disassembled along an indicated direction. For example, figure 6 (B) shows the first component is removed along the x-axis (i.e. (1, 0, 0), correlating to the x, y, and z axis respectively). The presenter in the video also states at TS 1:58: “The decoder is a recurrent neural network… to generate a sequence of components along with their direction.”). Digital_2021 does not teach “a regression technique”.
Zhu_2021, however, teaches “a regression technique” (Page. 653, Section 3.3: “… we propose spatial regression convolutional neural networks (SRGCNNs) as a deep learning paradigm to conduct spatial regression and prediction on spatial multivariate distributions”).
Digital_2021 and Zhu_2021 are analogous art because they are of the same field of endeavor called graph encoding/decoding. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Digital_2021 and Zhu_2021. The rationale for doing so would have been to utilize the regressive graph convolutional network (RGCN) model provided in Zhu_2021 to obtain directions to disassemble components of Digital_2021. Therefore, it would have been obvious to one of ordinary skill in the art to combine Digital_2021 and Zhu_2021 for the purposes of taking advantage of the predictive properties of the RGCN of Zhu_2021 (Page 653, Fig. 2). For example, the known set of data in Zhu_2021 (X) would be the edge relationships (i.e. connections) between the nodes of Digital_2021, with the values to be predicted (ŷ) being the direction for disassembly. Utilizing this method the training dataset provided in Digital_2021 (Fig. 5 and 7), which provides disassembly directions for training (Fig. 5), can be seen as the known set of training data (y) of Zhu_2021. Thus, the model would be trained on and determine directions of disassembly.
(3) Claims 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Digital_2021 in view of JP_2015 (JP 2015/522324 A) NOTE: JP_2015 is an English translation of a foreign patent. For purposes of examination, the examiner will refer to sections in the translation as “Page # Par. #”, where the Par. # correlates to the full paragraph on the page (e.g. “Page 3 Par. 2” would be the 2nd full paragraph on page 3).
Claim 6. Digital_2021 teaches the limitations of claim 1. Digital_2021 also teaches “the assembly graph includes a plurality of assembly nodes and a plurality of assembly edges, where each assembly node of the plurality of assembly nodes represents a particular component of the plurality of components (Fig. 3: Each node of the assembly graph (bottom left of the screen) is labeled as a separate component).
While Digital_2021 teaches an assembly graph with nodes and edges (the nodes labeled as components), Digital_2021 does not explicitly teach “a first assembly edge of the plurality of assembly edges represents a relationship between a first component represented by a first assembly node of the plurality of assembly nodes and a second component represented by a second assembly node of the plurality of assembly nodes, and wherein the first assembly node and the second assembly node are connected by the first assembly edge”.
JP_2015, however, teaches “a first assembly edge of the plurality of assembly edges represents a relationship between a first component represented by a first assembly node of the plurality of assembly nodes and a second component represented by a second assembly node of the plurality of assembly nodes (Page 3 Par. 1: “an assembly graph generation step (S80, S90) for expressing the relationship between the components in the assembly graph with the components as nodes and the adjacent relationships as edges”), and wherein the first assembly node and the second assembly nodes are connected by the first assembly edge.” (Page 3 Par. 1: “… a directed graph generation step (S60) for expressing as a connected graph a directed graph with components as nodes and a connected edge between components…”).
Digital_2021 and JP_2015 are analogous art because they are from the same field of endeavor called sequence generation. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine JP_2015 and Digital_2021. The rational for doing so would have been to clarify what the edges in Digital_2021 represent for use in the encoder to generate the graph embeddings. Therefore, it would have been obvious for one of ordinary skill in the art to utilize the knowledge from JP_2015, that edges represent adjacency relationships, and apply that to the node/edge graphs of Digital_2021. This would provide the data needed to produce the graph embeddings of Digital_2021.
Claim 7. Digital_2021 and JP_2015 teach the limitations of claim 6. JP_2015 also teaches “the relationship between first component and the second component indicates the first component is in contact with or connected to the second component (Page 3 Par. 1: “an assembly graph generation step (S80, S90) for expressing the relationship between the components in the assembly graph with the components as nodes and the adjacent relationships as edges”).
(4) Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Digital_2021 in view of JP_2015 in view of Bahubalendruni_2015 (“A Review on Graphical Assembly Sequence Representation Methods and Their Advancements”, Journal of Mechatronics and Automation, January 2015).
Claim 8. Digital_2021 and JP_2015 teach the limitations of claim 6. Digital_2021 further teaches “the one or more components graphs includes a first component graph corresponding to a first component of the plurality of components, wherein the first component graph comprises a plurality of component nodes and a plurality of component edges (Fig. 3: It is shown that the individual, color-coded, component graphs correlate to each individual component of the assembly graph. Each component graph is also shown to comprise of nodes and edges).
While Digital_2021 and JP_2015 teach component graphs, they do not teach “wherein each component node of the plurality of component nodes represents a portion of the first component, and wherein each edge of the first component graph represents a relationship between different portions of the first component corresponding to a pair of component edges of the component graph” (i.e. what the nodes and edges represent).
Bahubalendruni_2015, however, teaches “wherein each component node of the plurality of component nodes represent a portion of the first component (Page 19 Col. 2: “The nodes shown in Figure 6 represent set of parts…”) and wherein each edge of the first component graph represent a relationship between different portions of the first component (Page 19 Col. 2: “…the edges linking the nodes are the established liaisons based on contact between the components.”) corresponding to a pair of component edges of the component graph” (Page 19 Fig. 6).
Digital_2021, JP_2015, and Bahubalendruni_2015 are analogous art because they are from the same field of endeavor called sequence generation. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to apply the known technique of Bahubalendruni_2015 to the component graphs of Digital_2021 and JP_2015. The rationale for doing so would have been to provide the encoder of Digital_2021 and JP_2015 with the information required to generate the graph embeddings. Therefore, it would have been obvious to one of ordinary skill in the art to take the known liaison diagrams of Bahubalendruni_2015 and apply that knowledge to individual components of Digital_2021 and JP_2015 to create node/edge graphs of individual components, which would then the further utilized to create the graph embeddings of Digital_2021 and JP_2015.
Claim 9. Digital_2021, JP_2015, and Bahubalendruni_2015 teach the limitations of claim 8. Digital_2021 further teaches “the first component graph represents features of a particular assembly node of the assembly graph (Fig. 3: The component graphs can be seen to represent individual components of the assembly graph).”
(5) Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Digital_2021 in view of Marcheggiani_2017 (“Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling”, Conference on Empirical Methods in Natural Language Processing, 2017).
Claim 11. Digital_2021 teaches the limitations of claim 1. Digital_2021 further teaches “the decoder comprises a plurality of long-short term memory (LSTM) cells (Fig. 4: Multiple LSTM cells are represented), wherein a first cell of the plurality of LSTM cells is configured to determine a first component of the plurality of components to be removed during disassembly of the product (Fig. 4: The figure states “Decoder Architecture with attention layer for disassembly sequence generation) and to pass information identifying the first component … to a second cell of the plurality of LSTM cells (Fig. 4: The figure shows that information (i.e. C0, C1, etc.) on the components is passed to subsequent cells)”.
While Digital_2021 does show that additional information (i.e. h1, h2, etc.) is passed to subsequent cells, it does not explicitly teach that the additional information is “hidden state information”.
Marcheggiani_2017, however, teaches “hidden state information” (Page 1510, Section 4.2: “Formally, an LSTM can be defined as a function… that takes as input the sequence… and returns a hidden state hi …”).
Digital_2021 and Marcheggiani_2017 are analogous art because they are from the same field of endeavor called sequence generation. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Digital_2021 and Marcheggiani_2017. The rational for doing so would have been that it is known in the field that LSTM cells return hidden state information. Therefore, it would have been obvious to one of ordinary skill in the art to have the LSTM cells of Digital_2021 pass along the hidden state information of Marcheggiani_2017. This would allow the LSTM cells to obtain the required data needed to produce a sequence (i.e. disassembly sequence), obtaining the invention as specified in the claims.
(6) Claims 12-14 and 17, 19, 20 are rejected under U.S.C. 103 as being unpatentable over Digital_2021 in view of Ulianeko_2021 (US 2021/0173982 A1).
Claim 12. The majority of limitations of claim 12 are similar in scope to the limitations of claim 1, thus the rationale for rejection for claim 1 will apply to claim 12. The rejection rational for the further limitations will be provided below.
Digital_2021 does not teach “A system comprising: a memory, one or more processors communicatively coupled to the memory; an assembly descriptor executable by the one or more processors…”.
Ulianeko_2021, however, teaches “A system comprising: a memory, one or more processors communicatively coupled to the memory (Par. 33: “The present invention concerns… a computer system…”); an assembly descriptor executable by the one or more processors… (Par. 33: … a computer system… for automatically creating based on a computer-aided design (CAD) model comprising multiple parts forming an assembly…)”.
Digital_2021 and Ulianeko_2021 are analogous art because they are from the same field of endeavor called sequence planning. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Digital_2021 and Ulianeko_2021. The rationale for doing so would have been to improve the speed of disassembly generation of Ulianeko_2021. Therefore, it would have been obvious to on of ordinary skill in the art to apply the neural network of Digital_2021 to the system of Ulianeko_2021. Ulianeko_2021 states that “…from all test directions the test direction with the largest number of unblocked parts is selected as the disassembly direction.” Thus, new calculations must be made after every step of disassembly to find the new test direction with the largest number of unblocked parts. The time required for this process could be reduced by utilizing the neural network of Digital_2021 to predict directions of disassembly, thus improving the speed of the system of Ulianeko_2021.
Claim 13. The limitations of claim 13 are similar in scope to the limitations of claim 2, thus the rationale for rejection for claim 2 will apply to claim 13.
Claim 14. The limitations of claim 14 are similar in scope to the limitations of claim 5, thus the rationale for rejection for claim 5 will apply to claim 14.
Claim 17. The limitations of claim 17 are similar in scope to the limitations of claim 10, thus the rationale for rejection for claim 10 will apply to claim 17.
Claim 20. Ulianeko_2021 further details “A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations (Par. 41: “The present invention may further also provide a tangible non-transitory computer-readable data carrier comprising the CPP” NOTE: CPP equates to Computer Program Product (Par. 33)) for generating disassembly data” (Par. 44: “A disassembling sequence of steps may therefore be determined to sequentially remove parts from the assembly”).
(7) Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Digital_2021 in view of Ulianeko_2021 in view of JP_2015.
Claim 15. Digital_2021 and Ulianeko_2021 teach the limitations of claim 12. The limitations of claim 15 are similar in scope to the limitations of claims 6 and 7, thus the rationale for rejection for claims 6 and 7 will apply to claim 15.
(8) Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Digital_2021 in view of Ulianeko_2021 in view of JP_2015 in view of Bahubalendruni_2015.
Claim 16. Digital_2021, Ulianeko_2021, and JP_2015 teach the limitations of claim 15. The limitations of claim 16 are similar in scope to the limitations of claims 8 and 9, thus the rationale for rejection for claims 8 and 9 will apply to claim 16.
(9) Claims 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Digital_2021 in view of Ulianeko_2021 in view of Marcheggiani_2017.
Claim 18. Digital_2021 and Ulianeko_2021 teach the limitations of claim 12. The limitations of claim 18 are similar in scope to the limitations of claim 11, thus the rationale for rejection for claim 11 will apply to claim 18.
Claim 19. Digital_2021, Ulianeko_2021, and Marcheggiani_2017 teach the limitations of claim 18. Digital_2021 makes obvious “each LSTM cell of the plurality of LSTM cells is configured to receive attention weight information (Fig. 4) and to determine a next component to be removed as part of the disassembly sequence based at least in part on the attention weight information (TS 1:58: “The decoder is a recurrent neural network with attention mechanism to generate the sequence of components along with their direction.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sinha et al “A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications” discloses bilevel optimization and definitions along with providing figures displaying as much.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to ALEXANDER W LYON whose telephone number is (571)270-0757. The examiner
can normally be reached Monday-Thursday 8:30-5:30, Friday 8:30-12:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor,
Emerson Puente can be reached at (571) 272-3652. The fax phone number for the organization where
this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/AWL/
Examiner, Art Unit 2187
/EMERSON C PUENTE/Supervisory Patent Examiner, Art Unit 2187
Prosecution Insights