DETAILED ACTION
This action is responsive to the claims filed on February 7, 2024. Claims 1-27 are under examination.
Claims 1-27 are rejected under 35 USC 112(b)
Claims 1-3, 6, 8-12, 15, 17-21, 24, and 26-27 are rejected under 35 U.S.C. 103 as being unpatentable over Robison and Freymuth.
Claims 4-5, 7, 13-14, 16, 22-23, and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Robison, Freymuth, and Kumar.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-27 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “thin volumes” in the claims is a relative term which renders the claims indefinite. The term “thin volume” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
For purposes of examination, the term “thin volume” will be construed to mean a portion of an element that is below a predefined thickness. There does not appear to be explicit support for this in the specification, but it appears to be what the Applicant intended. To overcome this rejection, the Applicant will need to amend the claims to include a standard for what qualifies as thin volumes without adding new matter.
Dependent claims that depend from the rejected claims are rejected based on their dependences.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6, 8-12, 15, 17-21, 24, and 26-27: Robinson and Freymuth
Claims 1-3, 6, 8-12, 15, 17-21, 24, and 26-27 are rejected under 35 U.S.C. 103 as being unpatentable over NPL: “Automated mixed dimensional modelling from 2D and 3D CAD models” by Robinson et al. (Robinson) in view of NPL: “Swarm Reinforcement Learning for Adaptive Mesh Refinement” by Freymuth et al. (Freymuth).
Claims 1, 10, and 19
Regarding claim 10, Robinson teaches:
A system for reducing thin volume assemblies and generating sheet bodies , (Robinson Abstract “The procedures produce an accurate and efficient analysis model with little effort on the part of the user. The technique is applicable to thin walled components with local complex features and automatically creates analysis models where 3D elements representing the complex regions in the component are embedded in an efficient shell mesh representing the mid-faces of the thin sheet regions. As the resulting models contain elements of more than one dimension, they are referred to as mixed dimensional models.” – Robinson teaches a system for reducing thin volumes and generating sheet bodies.)
the system comprising: a storage device configured to store program instructions; and one or more processors operably connected to the storage device and configured to execute the program instructions to cause the system to: (Robinson Abstract “The motivation for this paper is to present procedures for automatically creating idealised finite element models from the 3D CAD solid geometry of a component.” Page 154 “The 3D MAT is still a topic of research in its own right; however, advances have been made in its computation in recent years [10]. The TranscenData implementation of the MAT [11], commercially available within the CADfix software, is advanced enough in both 2D and 3D to be used for the procedures presented in this paper.” – A computer is used, which includes processing and memory resources.)
establish thin volumes; (Robinson Page 156, 4.1. Identifying thin sheet regions “Thin sheet regions in solid bodies have large lateral dimensions relative to their thickness. The first step in the identification of thin sheets is to eliminate all medial faces that are definitely not thin. This is analogous to the medial edges which were eliminated in Section 3.1 in the description of dimensionally reducing 2D models.” – Thin volumes are determined.)
reducing the thin volumes; (Robinson Page 154, Left Column, Last Paragraph “The functionality required from the MAT to implement the dimensional reduction procedures described in this paper will now be summarised. The medial faces which represent the mid-surface of the thin sheet regions are required to represent the dimensionally reduced region. Where the MAT is not available it can be approximated by any mid-surface representation of the thin sheet regions in the 3D model. Other research that focuses on producing appropriate face representations for these types of regions includes [12,13].” Page 157, Right Column, Last Paragraph “After the insetting and rounding operations for a candidate medial face the inset edges and the splines are used to create edge loops within which mid-faces are created. The resulting mid-face for the profile in Fig. 11(a) is shown in Fig. 11(b). The mid-faces are sub-sets of the original medial faces in the thin region meaning no additional process is required to calculate their internal geometry.”
Robinson suggest automation of the reduction of thin volume process using software, and provides a few examples of software that can be used, but Robison does not appear to explicitly teach, but Robinson in view of Freymuth teaches:
A system for [mesh refinement] using reinforcement learning, (Freymuth Abstract “Adaptive Mesh Refinement (AMR) enhances the Finite Element Method, an important technique for simulating complex problems in engineering, by dynamically refining mesh regions, enabling a favorable trade-off between computational speed and simulation accuracy. […] We formulate AMR as a novel Adaptive Swarm Markov Decision Process in which a mesh is modeled as a system of simple collaborating agents that may split into multiple new agents.” – A system for mesh refinement using reinforcement learning.)
[…]
establish respective agents for [mesh elements]; (Freymuth Page 4, 3 Adaptive Swarm Mesh Refinement “In the following, we introduce the individual components of ASMR, including our novel Adaptive Swarm Markov Decision Process (ASMDP) and spatial reward function. We consider each element Ωti ∈ Ωt of a mesh Ωt to be an agent in a swarm system.” – Agents are assigned to mesh elements for mesh refinement.)
select actions for [mesh refinement] ; (Freymuth Page 4, 3 Adaptive Swarm Mesh Refinement “Here, S is the state space of the system, O is the space of observations for this state space, and A is the action space for the system of agents. Let SN ⊂ S, ON ⊂ O, AN ⊂ A denote the subsets of the state, observation, and action spaces with exactly N agents.”
assign initial rewards to the actions for each agent; execute the selected actions by the agents; (Freymuth Page 4, 3 Adaptive Swarm Mesh Refinement “The transition function T: SN × AN → SK maps to a new system state with a potentially different number of agents, and r : SN ×AN → RN is a per-agent reward function.” – Initial awards are assigned. The actions are executed in the transition.)
assign rewards to the selected actions based on a reward policy; and continue the reinforcement learning until at least one stopping criterion is met. (Freymuth Page 5, First-Second Paragraphs “The influence of each agent at step t on the reward, in terms of all successor agents it is responsible for up to step t + k, can then be computed via the matrix multiplication Mt,k := MtMt+1 . . .Mt+k−1. The usual objective in RL is to find a policy π : ON × AN → [0, 1] that maximizes the return, i.e., the expected discounted cumulative future reward [..]” – Rewards are dynamically assigned. The reinforcement learning stops when the model maximizes a specified return.)
It would have been obvious to a person of ordinary skill in the art to modify the mesh refinement of Robinson by the automated mesh refinement techniques of Freymuth because the person of ordinary skill in the art would be motivated by the aim of Robinson to efficiently and effectively automate the refinement of meshes, to look to Freymuth, which deploys efficient and robust refinement strategies.
(Robison Abstract “The motivation for this paper is to present procedures for automatically creating idealised finite element models from the 3D CAD solid geometry of a component. The procedures produce an accurate and efficient analysis model with little effort on the part of the user. The technique is applicable to thin walled components with local complex features and automatically creates analysis models where 3D elements representing the complex regions in the component are embedded in an efficient shell mesh representing the mid-faces of the thin sheet regions. As the resulting models contain elements of more than one dimension, they are referred to as mixed dimensional models.”; Freymuth Abstract “Adaptive Mesh Refinement (AMR) enhances the Finite Element Method, an important
technique for simulating complex problems in engineering, by dynamically
refining mesh regions, enabling a favorable trade-off between computational speed
and simulation accuracy. Classical methods for AMR depend on heuristics or
expensive error estimators, hindering their use for complex simulations. Recent
learning-based AMR methods tackle these issues, but so far scale only to simple
toy examples. We formulate AMR as a novel Adaptive Swarm Markov Decision
Process in which a mesh is modeled as a system of simple collaborating agents
that may split into multiple new agents. This framework allows for a spatial reward
formulation that simplifies the credit assignment problem, which we combine with
Message Passing Networks to propagate information between neighboring mesh
elements. We experimentally validate our approach, Adaptive Swarm Mesh Refinement
(ASMR), on challenging refinement tasks. Our approach learns reliable
and efficient refinement strategies that can robustly generalize to different domains
during inference. Additionally, it achieves a speedup of up to 2 orders of magnitude
compared to uniform refinements in more demanding simulations. We outperform
learned baselines and heuristics, achieving a refinement quality that is on par with
costly error-based oracle AMR strategies.”) Additionally/Alternatively, the use of Freymuth’s RL ASMR method is an obvious substitution for the software packages of Robinson (Page 154, Left Column, Third Paragraph After The Second Set of Bullet Points “The TranscenData implementation of the MAT [11], commercially
available within the CADfix software”) because Robison and Freymuth differ by the substitution of the software components of Robison and the RL ASMR methods of Freymuth, both functions, as demonstrated by the art of record, were known in the art, a person of ordinary skill in the art could have made the aforementioned substitution with predictable results.)
Claims 1 and 19 recite features substantially similar to the features of claim 10 and are rejected for at least the same reasons.
Claims 2, 11, and 20
Regarding claim 11, Robinson in view of Freymuth teaches the features of claim 10 and further teaches:
wherein the agents are responsible for selecting their actions and maintaining actions' reward history. (Freymuth Page 4, Last paragraph “The resulting framework is conceptually simpler than, e.g., a decentralized partially observable MDP with dummy states [29], makes the permutation equivariance of the agents explicit and naturally integrates both the agent-dependent reward and the mapping between agents. Formally, we define an Adaptive Swarm Markov Decision Process (ASMDP) as a tuple ⟨S,O,A, T, r, ξ,M⟩. Here, S is the state space of the system, O is the space of observations for this state space, and A is the action space for the system of agents. Let SN ⊂ S, ON ⊂ O, AN ⊂ A denote the subsets of the state, observation, and action spaces with exactly N agents. The transition function T : SN × AN → SK maps to a new system state with a potentially different number of agents, and r : SN ×AN → RN is a per-agent reward function. The observation graph of the agents is calculated from their states via the observation function ξ : SN → ON.” – The agents take action in the state space and their history is recorded and used to update the observation graph.)
Claims 2 and 20 recite features substantially similar to the features of claim 11 and are rejected for at least the same reasons.
Claims 3, 12, and 21
Regarding claim 12, Robinson in view of Freymuth teaches the features of claim 10, and further teaches:
wherein the actions include a reduce thin copy operation or a mid-surface operation. (Robison Abstract “The technique is applicable to thin walled components with local complex features and automatically creates analysis models where 3D elements representing the complex regions in the component are embedded in an efficient shell mesh representing the mid-faces of the thin sheet regions. As the resulting models contain elements of more than one dimension, they are referred to as mixed dimensional models.” – The aforementioned actions of Freymuth include the combined reducing a thin copy operation and a mid-surface operation of Freymuth.)
Claims 3 and 21 recite features substantially similar to the features of claim 12 and are rejected for at least the same reasons.
Claims 6, 15, and 24
Regarding claim 15, Robinson in view of Freymuth teaches the features of claim 10 and further teaches:
wherein stopping criteria include reaching a predefined number of iterations or achieving a specified level of convergence of rewards for the actions. (Freymuth Page 5, First-Second Paragraphs “The influence of each agent at step t on the reward, in terms of all successor agents it is responsible for up to step t + k, can then be computed via the matrix multiplication Mt,k := MtMt+1 . . .Mt+k−1. The usual objective in RL is to find a policy π : ON × AN → [0, 1] that maximizes the return, i.e., the expected discounted cumulative future reward [..]” – The reinforcement learning stops when the model maximizes a specified return of the rewards for the actions.)
Claims 6 and 24 recite features substantially similar to the features of claim 15 and are rejected for at least the same reasons.
Claims 8, 17, and 26
Regarding claim 17, Robinson in view of Freymuth teaches the features of claim 10 and further teaches:
wherein the actions comprise reduction actions and connection actions, and wherein the reduction actions are performed before the connection actions to ensure thin volume reduction before extending, imprinting and merging neighboring sheet bodies. (Robinson Page 158, 4.3 Creating the mixed dimensional model “The model is partitioned into thin sheet and complex regions by projecting the bounding edges of the newly created thin sheet mid-faces, outlined using the thick black line in Fig. 12(a), onto the defining faces of the medial faces representing the thin sheet region. The resulting projection lines are the dashed lines shown in Fig. 12(a). Faces are created through the body between the pairs of projected bounding edges on the opposing bounding faces. These faces, which are outlined using the dashed lines in Fig. 12(a), are called cutting faces in this paper and are used to partition the original solid. In the resulting model, Fig. 12(b), each thin sheet and complex region is represented by a separate body. To create the mixed dimensional model each body representing a thin sheet region is deleted and is represented by the thin sheet mid-surface, the creation of which is described above. For the model in Fig. 12 there is one thin sheet region and one detail feature, and its mixed dimensional model is shown in Fig. 12(c). As for the 2D case the diameter of the medial sphere used to trace the mid-face in the thin sheet region is used as the thickness attributes for the elements used to mesh it. In this paper the boundary between the body representing a detail feature, which consists of the cutting faces described earlier in the section, shaded dark grey in Fig. 12(c), and the bounding edges of the mid face which is shaded light grey in Fig. 12(c), is referred to as the mixed dimensional interface. Mixed dimensional coupling is required at the mixed dimensional interface to join the elements of different dimensionality together. Mixed dimensional coupling is described in more detail in Section 6.” Page 159 “There may also be scenarios where to further reduce the number of degrees of freedom in the finite element analysis model; the dimensionally reduced region is extended to the edge of the component. This is only appropriate if the adjacent region that is being incorporated into the thin sheet region is of similar thickness. In Fig. 8(e) the midline representing the thin sheet region has been extended to include the detail feature to the right of the component and in Fig. 12(d) the grey thin sheet region has been extended to the edge of the component on three sides. Clearly this is not appropriate on the side with the flange. Although this model will be more efficient than if these boundary regions were retained in full dimension, it should be noted that boundary conditions applied to the detail feature will have to be dimensionally reduced. Also, this type of reduction may not be feasible if the detail feature being removed is at the interface with other components.” - The actions of Freymuth agents include the Robison reduction actions and connection actions, Per Robison, the reduction actions are performed before the connection actions to ensure thin volume reduction before extending, imprinting and merging neighboring sheet bodies.)
Claims 8 and 26 recite features substantially similar to the features of claim 17 and are rejected for at least the same reasons.
Claims 9, 18, and 27
Regarding claim 17, Robinson in view of Freymuth teaches the features of claim 10 and further teaches:
wherein the actions are assigned rewards based on a reward policy that considers geometric integrity and connection relationships between the thin volumes and neighboring sheet bodies. (Freymuth Pages 5-6, Agents and Observations “The remesher may also refine unmarked elements to assert a conforming solution [41], i.e., to make sure that elements of the mesh align with each other at the boundaries and interfaces to ensure continuity of solution variables between adjacent elements. We define the mapping for an agent to its successor agents as the indicator function […]). In other words, an agent maps to all future agents that it spawns, or equivalently, an element is responsible for all sub-elements that it refines into over time. […] This reward function evaluates whether a refinement decreases the overall error by enough to justify the extra resources required, with a reward of 0 for unrefined elements. Thus, the reward maximizes” – The agents are responsible for ensuring geometric continuity/integrity of the meshed structure. Page error reduction, rather than simply encouraging a refinement of areas with a high existing error regardless of the resulting mesh improvement. Further, incorporating a novel area scaling term encourages the policy to focus on smaller elements with a high potential reduction in error rather than larger elements with a low average error reduction, up to some threshold depending on the element penalty α. We find that the combination of a local formulation and the area scaling term allows for a simpler credit assignment for the RL agents, as it ensures that each agent gets rewarded for its own actions and that rewards of elements of different sizes are on the same scale. – The agents are rewarded based on their actions, including ensuring geometric continuity/integrity of the meshed structure.)
Claims 9 and 27 recite features substantially similar to the features of claim 18 and are rejected for at least the same reasons.
Claims 4-5, 7, 13-14, 16, 22-23, and 25: Robinson, Freymuth, and Kumar
Claim(s) 4-5, 7, 13-14, 16, 22-23, and 25 are rejected under 35 U.S.C. 103 as being unpatentable over NPL: “Automated mixed dimensional modelling from 2D and 3D CAD models” by Robinson et al. (Robinson) in view of NPL: “Swarm Reinforcement Learning for Adaptive Mesh Refinement” by Freymuth et al. (Freymuth) and NPL: “Reward-Conditioned Policies” by Kumar et al. (Kumar).
Claims 4, 13, and 22
Regarding claim 13, Robinson in view of Freymuth teaches the features of claim 10, and further teaches:
using a supervised learning model for the actions. (Freymuth Page 3, Supervised Learning for AMR “Supervised Learning for AMR. Applications of supervised learning for AMR include directly calculating an error per mesh element with a Multilayer Perceptron (MLP) [11] and predicting mesh densities from domain images [70]. Additionally, recurrent networks have been used to find optimal marking strategies for second-order elliptical PDEs [74]. Another body of work speeds up the computation of Dual Weighted Residual [75, 76] error estimators by substituting expensive parts of the procedure with neural networks. Here, recent methods [77, 78] consider learning a metric tensor from solution information that can then be used in existing refinement procedures [24]. Other approaches [79, 13] employ neural networks to solve the strong form of the adjoint problem and use hand-crafted features to compute error estimates directly. We instead leverage the fact that RL can optimize non-differentiable rewards, enabling us to directly learn a refinement strategy in an iterative manner instead of learning error estimators or other specific facets of AMR.” – Freymuth teaches using supervised learning for any part of ASMR, and this can include a combination of the supervised learning for determining initial values for the reinforcement learning setup, including establishing initial rewards.)
Robinson in view of Freymuth does not appear to explicitly teach, but Robinson in view of Freymuth and Kumar teaches:
wherein the initial rewards are calculated using a supervised learning model for the actions. (Kumar Abstract “Reinforcement learning offers the promise of automating the acquisition of complex behavioral skills. However, compared to commonly used and well-understood supervised learning methods, reinforcement learning algorithms can be brittle, difficult to use and tune, and sensitive to seemingly innocuous implementation decisions. In contrast, imitation learning utilizes standard and well-understood supervised learning methods, but requires near-optimal expert data. Can we learn effective policies via supervised learning without demonstrations? The main idea that we explore in this work is that non-expert trajectories collected from suboptimal policies can be viewed as optimal supervision, not for maximizing the reward, but for matching the reward of the given trajectory. By then conditioning the policy on the numerical value of the reward, we can obtain a policy that generalizes to larger returns. We show how such an approach can be derived as a principled method for policy search, discuss several variants, and compare the method experimentally to a variety of current reinforcement learning methods on standard benchmarks.” Page 1, 1 Introduction “The main observation in our work is that any experience collected by an agent can be used as optimal supervision when conditioned on the quality of a policy. That is, actions that lead to mediocre returns represent “optimal” supervision for a mediocre policy. We can implement this idea in a practical algorithm by learning policies that are conditioned on the reward that will result from running that policy, or other quantities derived from the reward, such as the advantage value. In this way, all data gathered by the agent can be used as “optimal” supervision for a particular value of the conditioning return or advantage. – The rewards, including initial awards are calculated using supervised learning for the actions.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claims to modify the reward assignments of Freymuth by the supervised learning policy/award determinations for reinforcement learning of Kumar because the person of ordinary skill in the art would be motivated by the aim of Freymuth for efficient optimization of policies using reinforcement learning, to look to Kumar, which uses supervised learning to modify the policies/awards of a reinforcement learning model to obtain a policy that generalizes to larger returns. (Freymuth Abstract “We experimentally validate our approach, Adaptive Swarm Mesh Refinement (ASMR), on challenging refinement tasks. Our approach learns reliable and efficient refinement strategies that can robustly generalize to different domains during inference.”; Kumar Abstract “The main idea that we explore in this work is that non-expert trajectories collected from suboptimal policies can be viewed as optimal supervision, not for maximizing the reward, but for matching the reward of the given trajectory. By then conditioning the policy on the numerical value of the reward, we can obtain a policy that generalizes to larger returns.”)
Claims 4 and 22 recite features substantially similar to the features of claim 13 and are rejected for at least the same reasons.
Claims 5, 14, and 23
Regarding claim 14, Robinson in view of Freymuth teaches the features of claim 10, and further teaches:
wherein the actions are selected from known actions using a decision policy that allows for exploitation of a state space . (Freymuth Page 4, Last paragraph “The resulting framework is conceptually simpler than, e.g., a decentralized partially observable MDP with dummy states [29], makes the permutation equivariance of the agents explicit and naturally integrates both the agent-dependent reward and the mapping between agents. Formally, we define an Adaptive Swarm Markov Decision Process (ASMDP) as a tuple ⟨S,O,A, T, r, ξ,M⟩. Here, S is the state space of the system, O is the space of observations for this state space, and A is the action space for the system of agents. Let SN ⊂ S, ON ⊂ O, AN ⊂ A denote the subsets of the state, observation, and action spaces with exactly N agents. The transition function T : SN × AN → SK maps to a new system state with a potentially different number of agents, and r : SN ×AN → RN is a per-agent reward function. The observation graph of the agents is calculated from their states via the observation function ξ : SN → ON.” – The agents take action in the state space and their history is recorded and used to update the observation graph.)
Robinson in view of Freymuth does not appear to explicitly teach, but Robinson in view of Freymuth and Kumar teaches:
wherein the actions are selected from known actions using a decision policy that allows for exploitation of a state space utilizing supervised learning predictions to exploit knowledge gained from previous iterations of the supervised learning predictions. (Kumar Abstract “Reinforcement learning offers the promise of automating the acquisition of complex behavioral skills. However, compared to commonly used and well-understood supervised learning methods, reinforcement learning algorithms can be brittle, difficult to use and tune, and sensitive to seemingly innocuous implementation decisions. In contrast, imitation learning utilizes standard and well-understood supervised learning methods, but requires near-optimal expert data. Can we learn effective policies via supervised learning without demonstrations? The main idea that we explore in this work is that non-expert trajectories collected from suboptimal policies can be viewed as optimal supervision, not for maximizing the reward, but for matching the reward of the given trajectory. By then conditioning the policy on the numerical value of the reward, we can obtain a policy that generalizes to larger returns. We show how such an approach can be derived as a principled method for policy search, discuss several variants, and compare the method experimentally to a variety of current reinforcement learning methods on standard benchmarks.” Page 1, 1 Introduction “The main observation in our work is that any experience collected by an agent can be used as optimal supervision when conditioned on the quality of a policy. That is, actions that lead to mediocre returns represent “optimal” supervision for a mediocre policy. We can implement this idea in a practical algorithm by learning policies that are conditioned on the reward that will result from running that policy, or other quantities derived from the reward, such as the advantage value. In this way, all data gathered by the agent can be used as “optimal” supervision for a particular value of the conditioning return or advantage. – The rewards, including initial awards are calculated using supervised learning for the actions.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claims to modify the reward assignments of Freymuth by the supervised learning policy/award determinations for reinforcement learning of Kumar because the person of ordinary skill in the art would be motivated by the aim of Freymuth for efficient optimization of policies using reinforcement learning, to look to Kumar, which uses supervised learning to modify the policies/awards of a reinforcement learning model to obtain a policy that generalizes to larger returns. (Freymuth Abstract “We experimentally validate our approach, Adaptive Swarm Mesh Refinement (ASMR), on challenging refinement tasks. Our approach learns reliable and efficient refinement strategies that can robustly generalize to different domains during inference.”; Kumar Abstract “The main idea that we explore in this work is that non-expert trajectories collected from suboptimal policies can be viewed as optimal supervision, not for maximizing the reward, but for matching the reward of the given trajectory. By then conditioning the policy on the numerical value of the reward, we can obtain a policy that generalizes to larger returns.”)
Claims 5 and 23 recite features substantially similar to the features of claim 14 and are rejected for at least the same reasons.
Claims 7, 16, and 25
Regarding claim 16, Robinson in view of Freymuth teaches the features of claim 14, and further teaches:
wherein the supervised learning model used for predicting initial rewards employs features to improve prediction accuracy over iterations. (Kumar Page 1, Introduction “The main observation in our work is that any experience collected by an agent can be used as optimal supervision when conditioned on the quality of a policy. That is, actions that lead to mediocre returns represent “optimal” supervision for a mediocre policy. We can implement this idea in a practical algorithm by learning policies that are conditioned on the reward that will result from running that policy, or other quantities derived from the reward, such as the advantage value. In this way, all data gathered by the agent can be used as “optimal” supervision for a particular value of the conditioning return or advantage.” – The supervised learning model is used for predicting initial rewards employing features represented in the state space of Freymuth)
wherein the supervised learning model used for predicting initial rewards employs features including geometrythin volumes to improve prediction accuracy over iterations; (Robinson Abstract “The procedures produce an accurate and efficient analysis model with little effort on the part of the user. The technique is applicable to thin walled components with local complex features and automatically creates analysis models where 3D elements representing the complex regions in the component are embedded in an efficient shell mesh representing the mid-faces of the thin sheet regions. As the resulting models contain elements of more than one dimension, they are referred to as mixed dimensional models.” – Robinson teaches a system for reducing thin volumes in mesh geometry and generating sheet bodies.)
wherein the supervised learning model used for predicting initial rewards employs features including geometry and neighboring thin volumes to improve prediction accuracy over iterations; (Freymuth Page 4, 3 Adaptive Swarm Mesh Refinement “In the following, we introduce the individual components of ASMR, including our novel Adaptive Swarm Markov Decision Process (ASMDP) and spatial reward function. We consider each element Ωti ∈ Ωt of a mesh Ωt to be an agent in a swarm system. The agent’s state is its position in the mesh, as well as boundary conditions and other PDE-dependent quantities. The agents observation consists of a local view of a graph GΩt where each node represents a mesh element and each edge the neighborhood of two elements. – The rewards and policies are dependent upon all agents’ actions, including those of agents assigned as neighboring cells.)
Claims 7 and 25 recite features substantially similar to the features of claim 16 and are rejected for at least the same reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure
US 2021/0240887 A1 to Talha et al. (Teaches using mid-surfacing to defeature B-Rep models)
US 2004/0186604 A1 to Onodera et al. (Teaches producing shell model elements from 3B B-Spline elements)
US 2005/0073519 A1 to Onodera et al. (Teaches producing shell model elements from 3B B-Spline elements)
US 2024/0160800 A1 to Gromek et al. (Teaches a unified CAE and CAD simulation environment architecture)
US 2015/0339409 A1 to Sundaresh et al. (Teaches mid-surface defeaturing for CAD meshing)
US 2001/0037190 A1 to Jung (Teaches automated cost assignment to elements of CAD drawings)
NPL: “Distributed Task Allocation Algorithms for Multi-Agent Systems With Very Low Communication.” by Bapat et al. (Teaches dynamic assignment of agents to different tasks in reinforcement learning)
NPL: “Automatic solid decomposition and reduction for non-manifold geometric model generation” by Chong et al. (Teaches mesh decomposition, including defeaturing, such as mid-surfacing)
NPL: “Cubit 16.06 User Documentation” by Sandia National Laboratory (Teaches automated defeaturing by mid-surfacing and other surfacing simplifications)
NPL: “On the use of Machine Learning to Defeature CAD Models for Simulation” by Danglade et al. (Teaches using machine learning to defeature CAD models for meshing)
NPL: “Reinforcement Learning for Block Decomposition of CAD Models” by DiPrete et al. (Teaches reinforcement learning for decomposing CAD models into meshes for FEA)
NPL: “Dimensional Reduction of Surface Models for Analysis” by Donaghy et al. (Teaches defeaturing by dimensional reduction and reconciling mixed dimensions in the mode)
NPL: “DIVIDE-AND-CONQUER REINFORCEMENT LEARNING” by Ghosh et al. (Teaches dynamic assignment of agents in multi-agent systems)
NPL: “Viewpoint-independent Single-view 3D Object Reconstruction using Reinforcement Learning” by Ito et al. (Teaches 3D object reconstruction using RL)
NPL: “An integrated approach to realize multi-resolution of B-rep model” by Kim et al. (Teaches defeaturing for meshing)
NPL: “B-rep model simplification using selective and iterative volume decomposition to obtain finer multi-resolution models” by Kwon et al. (Teaches B-Rep dimensionality reduction/defeaturing, including mid-surfacing)
NPL: “Semantics-aware adaptive simplification for lightweighting diverse 3D CAD models in industrial plants” by Kwon et al. (Teaches simplification/defeaturing of 3D CAD models)
NPL: “3D reconstruction based on hierarchical reinforcement learning with transferability” by Lit et al. (Teaches using a multi-agent RL system for defeaturing 3D objects)
NPL: “Enhancing interactivity in structural optimisation through reinforcement learning: an application on shell structures” by Mirra et al. (Teaches shell structure optimization using RL)
NPL: “Human-level control through deep reinforcement learning” by Mnih et al. (Teaches automating human judgment using RL models)
NPL: “Defeaturing using Machine Learning” by Owen et al. (Inventor paper that uses ML to defeature models for meshing)
NPL: “CAD DEFEATURING USING MACHINE LEARNING.” by Owen et al. (Inventor paper that uses supervised learning ML models to defeature CAD models for meshing)
NPL: “Transformation of a thin-walled solid model into a surface model via solid deflation” by Sheen et al. (Teaches defeaturing by deflation, similar to surfacing)
NPL: “LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning” by Yang et al. (Teaches RL systems that divide tasks into sub-tasks and assign agents to each subtask, making sub-task-specific rewards)
NPL: “Simplifying Reinforced Feature Selection via Restructured Choice Strategy of Single Agent” by Zhao et al. (Teaches single-agent RL feature selection)
NPL: “Mid-surface abstraction for complex thin-wall models based on virtual decomposition” by Zhu et al. (Teaches mid-surface virtual decomposition of thin wall models.
(Contemporaneous/Non-Prior Art)
NPL: “Simplification of 3D CAD models with deep learning for augmented reality” by Arige (Teaches reducing complexity of 3D meshes for AR applications)
NPL: “CAD Dimensional Reduction for Shell Modeling using Reinforcement Learning” by Owens et al. (This is the white paper for this invention by the inventors)
NPL: “Reinforcement learning based automatic block decomposition of solid models for hexahedral meshing” by Zhang et al. (Teaches RL methods for mesh decomposition)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAY MICHAEL WHITE whose telephone number is (571)272-7073. The examiner can normally be reached Mon-Fri 11:00-7:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.M.W./Examiner, Art Unit 2188
/RYAN F PITARO/
Supervisory Patent Examiner, Art Unit 2188
