Prosecution Insights
Last updated: November 04, 2026
Application No. 18/143,660

REINFORCEMENT LEARNING-BASED DECISION OPTIMIZATION METHOD OF OILFIELD PRODUCTION SYSTEM

Non-Final OA §101
Filed
May 05, 2023
Priority
May 07, 2022 — CN 202210493119.1
Examiner
HALL, KRISTYN A
Art Unit
3676
Tech Center
3600 — Transportation & Electronic Commerce
Assignee
Beijing Zhongke Zhishang Technology Co. Ltd.
OA Round
1 (Non-Final)
82%
Grant Probability
Favorable
1-2
OA Rounds
0m
Est. Remaining
76%
With Interview

Examiner Intelligence

Grants 82% — above average
82%
Career Allowance Rate
629 granted / 767 resolved
+30.0% vs TC avg
Minimal -6% lift
Without
With
+-5.7%
Interview Lift
resolved cases with interview
Typical timeline
2y 2m
Avg Prosecution
22 currently pending
Career history
791
Total Applications
across all art units

Statute-Specific Performance

§101
4.5%
-35.5% vs TC avg
§103
43.5%
+3.5% vs TC avg
§102
19.0%
-21.0% vs TC avg
§112
27.7%
-12.3% vs TC avg
Black line = Tech Center average estimate • Based on career data from 767 resolved cases

Office Action

§101
DETAILED ACTION The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Priority Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Step 1 of the Subject Matter Eligibility Test entails considering whether the claimed subject matter falls within the four statutory categories of patentable subject matter identified by 35 U.S.C. 101: Process, machine, manufacture, or composition of matter. Claims 1-9 are directed to a method (process), a system (machine or manufacture), and a non-transitory medium (manufacture), respectively. As such, the claims are directed to statutory categories of invention. If the claim recites a statutory category of invention, the claim requires further analysis in Step 2A. Step 2A of the Subject Matter Eligibility Test is a two-prong inquiry. In Prong One, examiners evaluate whether the claim recites a judicial exception. Claim 1 recites abstract limitations, including: “constructing an evaluation function for production optimization of a gas injection reservoir; establishing, during a process of production optimization, an enforced constraint model based on input parameters and a boundary constraint condition; with the constraint model and the boundary constraint condition as constraints, the reservoir injection-production system surrogate model as a basis, and the evaluation function as an optimization direction, searching reservoir production optimization schemes as optimal production schemes; and selecting one optimal production scheme from the optimal production schemes according to actual requirements, obtaining an injection scheme corresponding to the one optimal production scheme.” These limitations, as drafted, are a process that, under its broadest reasonable interpretation, represent mathematical relationships, mathematical formulas or equations, and/or mathematical calculations and are therefore mathematical concepts. The mere recitation of a generic computer does not take the claim out of the mathematical concepts grouping. Thus, the claim recites an abstract idea. If the claim recites a judicial exception in step 2A Prong One, the claim requires further analysis in step 2A Prong Two. In step 2A Prong Two, examiners evaluate whether the claim recites additional elements that integrate the exception into a practical application of that exception. The claim recites the additional elements of: collecting on-site dynamic production data to establish a data cube for reservoir production optimization; training a preset machine learning model based on the data cube to obtain a reinforcement learning-based reservoir injection-production system surrogate model configured to predict oil production according to the on-site dynamic production data; and controlling injection well equipment for injecting according to the injection scheme “[C]ollecting on-site dynamic production data” amounts to insignificant pre-solution activity (i.e., activity incidental to the primary process/product that is merely a nominal or tangential addition to the claim, see MPEP 2106.05(g)). The training of the machine learning model is recited at a high-level of generality and is merely invoked as tools to perform the abstract idea (i.e., “apply it”). “[C]ontrolling injection well equipment for injecting according to the injection scheme” amounts to no more than instructions to apply the exception (i.e., “apply it”). Accordingly, in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. If the additional elements do not integrate the exception into a practical application in step 2A Prong Two, then the claim is directed to the recited judicial exception, and requires further analysis under Step 2B to determine whether they provide an inventive concept (i.e., whether the additional elements amount to significantly more than the exception itself). As discussed above, collecting data is considered insignificant extra-solution activity as the limitation does not amount to more than mere data gathering. Given the generality of the data collection, the limitation does not contain significantly more to provide a practical application (see MPEP 2106.05(g)). As discussed above, the additional element of training the machine learning model is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception. Mere instructions to apply an exception using generic computer components/functions cannot provide an inventive concept (MPEP 2106.05(f)). “[C]ontrolling injection well equipment for injecting according to the injection scheme” merely amount to “apply it”. The reciting of claim limitations that attempt to cover any solution (i.e., controlling injection well equipment) to an identified problem (i.e., optimizing oilfield production) with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result (i.e., what aspects are changed or how the change is affected by the abstract idea) does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words “apply it” (see MPEP 2106.05(f)(1)). Claim 2-3 and 5-8 further recite: Claim 2: after obtaining the reservoir injection-production system surrogate model, the method further comprises: based on the reservoir injection-production system surrogate model, quantitatively characterizing connectivity between an injection well and a production well; Claim 3: said quantitatively characterizing connectivity between an injection well and a production well specifically comprises: calculating importance of each input variable of the reservoir injection-production system surrogate model to the model, namely, randomly changing the sequence of data of a certain variable in a data set while keeping other variables constant so as to construct a new training set, training the model with the new data set, and calculating an error of a corresponding prediction result; calculating a difference value between a prediction error of the model trained based on the 14 new data set and a prediction error of a model trained based on an original data set as sensitivity of a corresponding variable in a data set to a model; and calculating the connectivity between an injection well and a production well through the following formula: L j = E I V j + E I P j + E I M j ∑ i = 1 4 E I V i + E I P i + E I M i wherein L j denotes a connectivity factor between a jth injection well and a production well, E I V j denotes sensitivity of a gas injection volume of the jth injection well to a model, E I P j denotes sensitivity of injection pressure of the jth injection well to the model, and E I M j denotes sensitivity of an injection mode of the jth injection well to the model; Claim 4: said training a preset machine learning model based on the data cube to obtain a reinforcement learning-based reservoir injection-production system surrogate model configured to predict oil production according to the on-site dynamic production data specifically comprises: extracting from the data cube a data set required for model training, and dividing the data set into training sets and test sets in a ratio of 9:1, wherein in the data set, the well status of shut-in and producing and choke size of the production wells, as well as the gas injection volume, injection pressure and injection mode of the injection wells are taken as inputs of the model, and oil production, gas production and gas-oil ratio of the production wells are taken as outputs of the model; constructing a deep neural network-based (DNN-based) machine learning model, wherein the machine learning model has 3 hidden layers, with each layer having 60 neurons; and training the machine learning model by adopting the training sets, and testing the trained machine learning model by adopting the test sets to obtain the reinforcement learning-based reservoir injection-production system surrogate model; Claim 6: the evaluation function is expressed as follows: O x = e x p Q o f D , W , θ G O R f D , W , θ + 1 wherein O x denotes an evaluation function, Q o f D , W , θ denotes oil production, and G O R f D , W , θ denotes agas-oil ratio; Claim 7: said establishing an enforced constraint model based on input parameters and a boundary constraint condition specifically comprises: establishing a physical constraint model between injection volume and injection pressure, that is, constructing, with injection volume as an input and injection pressure as an output, and by a machine learning model, an intelligent constraint model S which can predict injection pressure with the injection volume, wherein a relationship between the injection volume and injection pressure is represented as follows: I P p r e d w , t = S I V w , t , W , θ wherein I P p r e d w , t denotes an injection pressure prediction value of a wth injection well at moment t, I V w , t denotes an injection volume of the wth injection well at moment t, W denotes a weight among neurons of a machine learning model, and θ denotes a threshold value in the neurons; besides, a following boundary constraint condition is set for input variables corresponding to the injection well: I V w , t ∈ a * A v e I V ,   b * M a x I V , I P w , t ∈ M i n I P , M a x I P wherein a and b are constraint factors, which respectively have a value range a ∈ 0,1 and b ∈ 0.5,2 ; I P w , t denotes injection pressure of a wth injection well at moment t; A v e I V denotes an average value of injection volume, M a x I V denotes a maximum value of injection volume, M i n I P denotes a minimum value of injection pressure, and M a x I P denotes a maximum value of injection pressure; and in addition, a boundary constraint condition of a choke size in production measures is expressed as: C S t ∈ 0 , * A V E C S + c * M A X C S - M I N C S wherein C S t denotes a choke size of a production well at moment t; A V E C S denotes an average value of choke size; c denotes a flow coefficient, M A X C S denotes a maximum value of choke size, and M I N C S denotes a minimum value of choke size; and Claim 8: said with the constraint model and the boundary constraint condition as constraints, the reservoir injection-production system surrogate model as a basis, and the evaluation function as an optimization direction, searching reservoir production optimization schemes for an optimal production scheme specifically comprises: establishing an injection-production optimization model based on a particle swarm optimization algorithm, wherein inputs of the injection-production optimization model comprise injection volume, injection pressure and injection mode of each injection well, and outputs comprise oil production and gas-oil ratio of a target well; with a relation model between the injection volume and injection pressure and a boundary constraint condition as constraints, the reinforcement learning-based reservoir injection- production system surrogate model as a basis, and the evaluation function as an optimization direction, searching reservoir production optimization schemes; and optimizing different injection schemes, calculating, based on the optimized injection scheme, oil production and gas-oil ratio of a production well under corresponding conditions by using the injection-production system surrogate model, searching for Pareto front aiming at reservoir production optimization to obtain the optimal production schemes which merely narrows the previously recited abstract idea limitations. Claim 4 recites the additional elements of collecting dynamic production data from reservoir production wells and neighboring injection wells. The collecting amounts to insignificant extra-solution activity (i.e., activity incidental to the primary product/process that is merely a nominal or tangential addition to the claim, see MPEP 2106.05(g)) as Salino (US 2017/0044877 see ¶ [0050]) discloses sensors for data collection in production wells and injection wells are well-known, routine, and conventional in the art. Claim 4 further recites: the dynamic production data of the production wells comprise: oil production, gas production, gas-oil ratio, well status of shut-in and producing, and choke size, and the dynamic production data of the injection wells comprise: gas injection volume, injection pressure, and injection mode; and establishing the data cube for reservoir production optimization by using the collected dynamic production data which merely narrows the previously recited abstract idea limitations. Claim 9 recites the additional elements of the injection well equipment comprises a single well distributor, an injection pump and injection facility, and the injection facility comprises oil pipe inlet valve, oil pipe outlet valve and main valve; and distributing, by the single well distributor, gas according to the gas injection volume, controlling the injection pump based on the injection pressure, and controlling the injection facility according to the injection mode. “[T]he injection facility comprises oil pipe inlet valve, oil pipe outlet valve and main valve” amounts to merely indicating a field of use or technological environment in which to apply a judicial exception and cannot integrate the judicial exception into a practical application (see MPEP 2106.05(h)). “[D]istributing, by the single well distributor, gas according to the gas injection volume, controlling the injection pump based on the injection pressure, and controlling the injection facility according to the injection mode” merely amount to “apply it”. The reciting of claim limitations that attempt to cover any solution (i.e., gas distribution) to an identified problem (i.e., optimizing operations) with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result (i.e., what aspects are changed or how the change is affected by the abstract idea) does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words “apply it” (see MPEP 2106.05(f)(1)). Claim 9 also further recites the injection scheme comprises a gas injection volume, an injection pressure and an injection mode, the injection well equipment comprises a single well distributor, an injection pump and injection facility, and the injection facility comprises oil pipe inlet valve, oil pipe outlet valve and main valve; wherein the reinforcement learning-based decision optimization method further comprises: selecting the one optimal production scheme from the Pareto front according to the actual requirements; obtaining the injection scheme corresponding to the one optimal production scheme; and distributing, by the single well distributor, gas according to the gas injection volume, controlling the injection pump based on the injection pressure, and controlling the injection facility according to the injection mode which merely narrows the previously recited abstract idea limitations. Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Mustapha (US 2024/0110469) discloses reservoir modeling and optimization using machine learning (Fig. 5). Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTYN A HALL whose telephone number is (571)272-8384. The examiner can normally be reached M-F 9:00-5:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nicole Coy can be reached at (571) 272-5405. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /KRISTYN A HALL/Primary Examiner, Art Unit 3672
Read full office action

Prosecution Timeline

May 05, 2023
Application Filed
Sep 30, 2026
Non-Final Rejection mailed — §101 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12768262
APPARATUS AND METHODS FOR UNINTERRUPTED DRILLING
1y 10m to grant Granted Oct 20, 2026
Patent 12760607
SEMI-AUTOMATIC DIRECTIONAL DRILL AND METHOD
2y 2m to grant Granted Oct 13, 2026
Patent 12760642
Method for Determining Geology While Drilling
1y 9m to grant Granted Oct 13, 2026
Patent 12753656
DRILLING MACHINE ATTACHMENT FOR LOADER
1y 9m to grant Granted Oct 06, 2026
Patent 12747641
ELECTRO-MECHANICAL DISCONNECT MECHANISM
1y 10m to grant Granted Sep 29, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
82%
Grant Probability
76%
With Interview (-5.7%)
2y 2m (~0m remaining)
Median Time to Grant
Low
PTA Risk
Based on 767 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month