Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Songhori et al. [US 12,488,164 B2] in view of Ren [US 2022/0292335 A1].
Taking claim 1 as exemplary of claims 1 and 14, Songhori et al. teach a computer-implemented processing system [FIG. 7], comprising:
a memory configured to store a reinforcement learning agent [FIG. 1 element 110, column 1, line 29 deep reinforcement learning (RL) agent] and a solver module [FIG. 1 element 130, column 7, lines 41-48 greedy legalization algorithm solves to a legal position, simulated annealing and hill climbing are optimization techniques used as types of solvers]; and
one or more processors operatively coupled to the memory, the one or more processors being configured to:
implement an iterative reinforcement learning training process for an integrated circuit to train the reinforcement learning agent [column 11, line 56-column 12, line 14 trains the node placement model on training data using reinforcement learning], in which the reinforcement learning agent learns an ordering of transistors for the integrated circuit [column 7, lines 36-40 the initial chip placement 122 is interpreted as an ordering] by placement of one node on an encoded grid [column 5, lines 56-60 NxM grid, column 10, lines 25-37 canvas, encoder] per iteration [column 6, lines 11-67 at each time step], in which the reinforcement learning agent is configured to iterate until all nodes for the integrated circuit are placed on the encoded grid [column 11, lines 56-58 generate high quality placements];
upon placement of all the nodes on the encoded grid [the initial chip placement], implement the solver module using the ordering of the nodes as an input [column 7, lines 37-40 provide the initial chip placement 122 to the legalization engine 130], the solver module being configured to perform an optimization [column 7, lines 37-40 adjusts the initial chip placement, column 7, lines 41-58 refine the initial chip placement, evaluation and fine-tuning] to minimize spacing between the nodes [column 7, lines 41-46 generate a legalized placement honoring minimum spacing constraints]; and
save the trained reinforcement learning agent in the memory [it follows that a reinforcement learning agent stored in a memory (see above) would save the trained accordingly, column 16, lines 65-67 capable of storing information].
However, Songhori et al. do not appear to teach by placement of one transistor but rather of nodes that represent components [column 5, lines 30-32].
Ren teaches a computer-implemented processing system [0013, 0180] comprising:
a memory configured to store a reinforcement learning agent [1804, 1320]; and
one or more processors operatively coupled to the memory [1702, 1302], the one or more processors being configured to:
implement an iterative reinforcement learning training process for an integrated circuit to train the reinforcement learning agent [0042 RL placer, 0047, 0082], in which the reinforcement learning agent learns an ordering of transistors for the integrated circuit by placement of one transistor on an encoded grid per iteration [0075 placement routine executes moves on a placement template, 0079 grid to be routed in a next iteration, moves specify placement order, 0042 for each placement step], in which the reinforcement learning agent is configured to iterate until all transistors for the integrated circuit are placed on the encoded grid [Abstract generates circuit layouts for a library, leverages RL to generate device placements in the layouts, 0049-0050, 0072 placement process concludes when all devices are placed, multiple procedures may be run, the final layout achieved the highest reward, 0074-0075 scoring function is optimized].
Thus, a person having ordinary skill in the art to which the claimed invention pertains would have found it obvious to consider a node as a transistor because transistors are components in the netlist data and nodes can represent a single component [Songhori et al. column 4, lines51-57, column 53-55].
Therefore, the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains because a reinforced learning-based device placement algorithm (i.e. and RL placer) may be implemented with generate placements with accuracy [0042].
Taking claim 2 as exemplary of claims 2 and 15, wherein the one or more processors are further configured to generate an integrated circuit design according to the optimization [either reference can be considered to generate an IC design as part of the above citations; see, also, Songhori et al. column 7, line 59-column 8, line 3].
Taking claim 3 as exemplary of claims 3 and 16, wherein the reinforcement learning agent learns the ordering of transistors for the integrated circuit by placing either the one transistor on the encoded grid per iteration or by placing a pair of complementary transistors on the encoded grid per iteration [Ren 0042 the RL placer will place a pair].
Taking claim 4 as exemplary of claims 4 and 17, wherein the reinforcement learning agent employs a policy proximal optimization according to an RL action space [Songhori et al. column 14, line 61-column 15, line 3; Ren 0073, 0079].
As per claim 5, wherein the policy proximal optimization implements a probability distribution for every transistor for where that could be placed on the encoded grid [Songhori et al. column 6, lines 40-50; Ren 0072].
Taking claim 6 as exemplary of claims 6 and 18, wherein the one or more processors are further configured to implement a router module after one or more intermediate iterations of the iterative reinforcement learning training process [Songhori et al. column 13, lines 9-25; Ren 0099].
Taking claim 7 as exemplary of claims 7 and 19, Songhori et al. do not teach wherein the solver module implements at least one of a Boolean Satisfiability solver, a Satisfiability Modulo a Theory (SMT) solver, or a Mixed- Integer Linear Programming (MILP) solver. Ren teaches SAT and MILP as old and well-known solver modules. Thus, a person having ordinary skill in the art to which the claimed invention pertains would have found it obvious to use SAT, SMT, or MILP as the solver module by considering the particular software or tool available to the user. Therefore, the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains because implementing a SAT, SMT, or MILP solver is a design choice.
Taking claim 8 as exemplary of claims 8 and 20, wherein the reinforcement learning agent learns the ordering of transistors according to actions, states and rewards for each iteration, in which each state includes connectivity and coordinates of previously placed transistors on the encoded grid [actions, states and rewards are intrinsic and fundamental components of reinforcement learning].
As per claim 9, wherein the reward at an end of each iteration is calculated as a linear combination of cell area, wirelength, and any routability or timing penalty [Songhori et al. column 3, lines 48-51, column 4, lines 13-25, column 12, lines 34-41; Ren 0054, 0065, 0075].
As per claim 10, wherein cell area is defined by a minimum bounding box that includes all currently placed transistors at a given iteration [Songhori et al. column 12, lines 65-66, column 13, line 23; Ren 0102 box tensor].
As per claim 11, wherein the minimum bounding box represents a half-perimeter wire length [Songhori et al. column 12, lines 65-66].
As per claim 12, wherein the reward at a conclusion of a final iteration is back-propagated through the reinforcement learning agent [Songhori et al. column 12, lines 1-4 supervised learning; Ren 0049 feedback].
As per claim 13, wherein routability at each iteration is approximated using at least one of congestion, pin density, area or wire length [Songhori et al. column 4, lines 13-25, column 12, line 34-column 13, line 50; Ren 0049, 0054].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
C.-K. Cheng et al. disclose “Machine Learning Prediction for Design and System Technology Co-Optimization Sensitivity Analysis” (entire document); C.-K. Cheng et al. disclose “Complementary-FET (CFET) Standard Cell Synthesis Framework for Design and System Technology Co-Optimization Using SMT” (entire document); D. Lee et al. disclose “SP&R: SMT-Based Simultaneous Place-and-Route for Standard Cell Synthesis of Advanced Nodes” (entire document); A. Mirhoseini et al. disclose “Chip Placement with Deep Reinforcement Learning” (Abstract, Figure 1); Matei et al. [US 2024/0160802 A1] disclose reinforcement learning and optimization (entire document); Martinello et al. [US 2025/0077758 A1] disclose placement using SMT (Abstract); Ushijima-Mwesigwa et al. [US 2024/0330698 A1] disclose combining reinforcement learning and a solver (entire document).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEIGH M GARBOWSKI whose telephone number is (571)272-1893. The examiner can normally be reached M-F 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jack Chiang can be reached at 571-272-7483. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEIGH M GARBOWSKI/ Primary Examiner, Art Unit 2851
Prosecution Insights