DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
2. This office action is in response to Amendments and Remarks filed on 04/27/2026 for application number 18/931,378 filed on 10/30/2024, in which claims 1-20 were previously presented for examination.
3. Claim(s) 10-11 has/have been canceled, and claim(s) 1, 12, and 20 has/have been amended. Accordingly, claim(s) 1-9, and 12-20 is/are currently pending.
Priority
4. Acknowledgment is made that Applicant has not claimed any foreign or domestic priority.
Examiner Notes
5. The Examiner has cited particular paragraphs or columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. The prompt development of a clear issue requires that the replies of the Applicant meet the objections to and rejections of the claims. Applicant should also specifically point out the support for any amendments made to the disclosure (see MPEP §2163.06). Applicant is reminded that the Examiner is entitled to give the Broadest Reasonable Interpretation (BRI) of the language of the claims. Furthermore, the Examiner is not limited to Applicant’s definition which is not specifically set forth in the claims. SEE MPEP 2141.02 [R-07.2015] VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, INCLUDING DISCLOSURES THAT TEACH AWAY FROM THE CLAIMS: A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
Response to Arguments
6. Applicant's arguments filed 04/27/2026 have been fully considered but they are not persuasive.
Rejections under 35 U.S.C. 101
7. In regard to claims 1 and 12, Applicant has amended the claims to recite the limitation of “the UAS performs the action based on the command” in claim 1 and the limitation of “performing the action by the UAS in response to the command” in claim 12. These limitations integrate their respective claim into a practical application. As such, the rejection of the claims under 35 U.S.C. 101 for claims 1 and 12 and their dependent claims is withdrawn.
8. In regard to claim 20, Applicant argues claim 20 recites “defining a transition function using a first helper function to generate a probability tensor having a first number of discrete values within a first matrix of state features.” Claim 20 also recites “defining a reward function using a second helper function to generate a reward tensor having a second number of discrete values within a second matrix of state features.” Claim 20 also recites “using an MDP solver located on the UAS to generate the MDP policy based on the set of finite states, the set of actions, the probability tensor, and the reward tensor, wherein the MDP solver reduces the first number of discrete values of the probability tensor and the second number of discrete values of the reward tensor to a number of values for the set of actions for the UAS.” Applicant submits that these limitations are not directed to a mental process or a mathematical concept. Applicant continues generating a probability tensor having a matrix of state features and a reward tensor having a matrix of state features cannot be practically performed in the human mind. The tensors are data matrices that readable by the MDP solver to then be reduced to the MDP policy, which has a reduced number of values compared to the tensors. The data matrices provided to the MDP solver are not readable or able to be recognized by the human mind. Thus, the human mind cannot reduce the number of discrete values within the matrices of the tensors as the human mind cannot read them. Thus, claim 20 is not directed to a mental process.
9. However, the courts consider a mental process (thinking) that “can be performed in the human mind, or by a human using a pen and paper” to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). Examiner asserts, defining a set of finite states of a finite state machine, including defining a transition function using a first helper function to generate a probability tensor having a first number of discrete values within a first matrix of state features, is a mental process that a human is capable of doing so by using pen and paper. Defining a reward function, which is an equation that accepts some inputs and outputs the results, is also a mental process which mathematicians have been doing, even before the invention of computers. Furthermore, using an MDP solver […] to generate the MDP policy based on the set of finite states […] is also using the mathematical equations of the MDP solver to calculate the result. A human is fully capable of generating a probability tensor having a matrix of state features and a reward tensor having a matrix of state features by using pen and paper. In regard to applicant’s assertion that the data matrices provided to the MDP solver are not readable or able to be recognized by the human mind, Examiner asserts, while such requirements are not a consideration for determining the eligibility of a claim under 35 U.S.C. 101, a human is capable of creating and performing operation on matrices. Accordingly, claim 20 recites at least one abstract idea and it is ineligible.
10. Applicant continues, claim 20 also is not directed to a mathematical concept. According to MPEP 2106.04(a)(2)(I), “[a] claim does not recite a mathematical concept (i.e., the claim limitations do not fall within the mathematical concept grouping), if it is only based on or involves a mathematical concept.” Here, although the limitation that “the MDP solver reduces the first number of discrete values of the probability tensor and the second number of discrete values of the reward tensor to a number of values for the set of actions for the UAS” may involve a mathematical concept, it does not recite a mathematical concept. Instead, claim 20 recites a limitation of reducing the discrete values in the tensors, in the form of matrices, to a smaller number of discrete values to optimize operations in guiding the UAS. Thus, this feature is not directed to a mathematical concept.
11. However, the quoted section of MPEP above, is not relevant to claim 20. Indeed, the same section of MPEP clarifies that “determining that the claims to a particular configuration of inertial sensors and a particular method of using the raw data from the sensors in order to more accurately calculate the position and orientation of an object on a moving platform did not merely recite “the abstract idea of using ‘mathematical equations for determining the relative position of a moving object to a moving reference frame’.” (See at least MPEP 2106.04(a)(2)(I)). Furthermore, the same section of MEPE states: “It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula." In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). See, e.g., SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 127 USPQ2d 1597, 1599 (Fed. Cir. 2018) (holding that claims to a ‘‘series of mathematical calculations based on selected information’’ are directed to abstract ideas); Digitech Image Techs., LLC v. Elecs. for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (holding that claims to a ‘‘process of organizing information through mathematical correlations’’ are directed to an abstract idea); and Bancorp Servs., LLC v. Sun Life Assurance Co. of Can. (U.S.), 687 F.3d 1266, 1280, 103 USPQ2d 1425, 1434 (Fed. Cir. 2012) (identifying the concept of ‘‘managing a stable value protected life insurance policy by performing calculations and manipulating the results’’ as an abstract idea)” (See at least MPEP 2106.04(a)(2)(I)), which is relevant to claim 20. As such, claim 20, which specifically uses MDP and performs operations on tensors (Matrices), recites a mathematical concept and it is ineligible.
12. Applicant further argues, even if the claims are directed to an abstract idea, which Applicant does not concede, the claims integrate any potential abstract idea into a practical application. In determining this step, according to MPEP 2106.04(d)(I), an improvement in the functioning of a computer, or an improvement to other technology or technical field is an integration of the abstract idea into a practical application. Here, the claims are directed to instructing the UAS to perform an action in response to a threat, as recited in the claims. The action is performed without the need for operator intervention. Thus, any possible abstract idea is integrated into a practical application.
13. However, a practical application has positive recitation of control, which cannot be found in claim 20. While Applicant asserts the claims are directed to instructing the UAS to perform an action in response to a threat, claim 20 lacks such a limitation. Indeed amending the claim by adding a similar limitation as above, depending on the language, may overcome the rejection of the claim. Furthermore, Applicant asserts the invention improves upon conventional functioning of a computer, or upon conventional technology or technological processes. However, the court has decided the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)) (See MPEP 2106.05(a)). Applicant has failed to point out the additional elements in the claim pertaining to the improvement in the functioning of a computer, or an improvement to other technology or technical field. While Applicant asserts performing an action in response to a threat without the need for operator intervention, which is not recited in claim 20, is an improvement in the functioning of a computer, or an improvement to other technology, the capability of taking an action to avoid a threat or a risk is an integral part of autonomous navigation of unmanned vehicles and drones and it is far from an improvement in the functioning of a computer, or an improvement to other technology. Accordingly, Applicant only provides conclusory statements.
14. As such, these arguments are unpersuasive and the rejection of claim 20 is maintained.
Rejections under 35 U.S.C. 102
15. In regard to claim 20, Applicant’s arguments and amendments have been addressed in the new rejection outlined below.
16. Applicant’s arguments with respect to claim(s) 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Rejections under 35 U.S.C. 103
17. In regard to claims 1, and 12, Applicant’s arguments and amendments have been addressed in the new rejection outlined below.
18. Applicant’s arguments with respect to claim(s) 1, and 12 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
19. Applicant argues dependent claim(s) is/are patentable by the virtue of their dependency on one of the independent claims and the additional features recited in the dependent claims.
20. This argument is unpersuasive as each independent claim and dependent claim has been fully rejected and for the reasons given above.
Claim Rejections - 35 USC § 101
21. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
22. Claim(s) 20
is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
23. The determination of whether a claim recites patent ineligible subject matter is a 2 step inquiry.
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), see MPEP 2106.03, or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis: see MPEP 2106.04
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon? see MPEP 2106.04(II)(A)(1)
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? see MPEP 2106.04(II)(A)(2) and 2106.05(a) thru (d) for explanations.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? see MPEP 2106.05
101 Analysis – Step 1
24. Claim(s) 20 is/are directed to a method (i.e. a process).
Therefore, claim(s) 20 is/are within at least one of the four statutory categories.
101 Analysis – Step 2A, Prong I
25. Regarding Prong I of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether they recite subject matter that falls within one of the follow groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes. see MPEP 2106(A)(II)(1) and MPEP 2106.04(a)-(c).
26. Independent claim(s) 20 include(s) limitations that recite an abstract idea (emphasized below [with the category of abstract idea in brackets]) and the additional limitations beyond the abstract idea are as follows (where the underlined portions are the “additional limitations” [with a description of the additional limitations in brackets], while the bolded portions continue to represent the “abstract idea”.)
27. Claim 20 recites:
A method for generating a Markov Decision Process (MDP) policy [mathematical concepts] for use in a route survivability autonomous system for an unmanned aircraft system (UAS), the method comprising:
defining a set of finite states for the UAS [mental process/step];
defining a set of actions for the UAS corresponding to the set of finite states [mental process/step];
defining a transition function using a first helper function to generate a probability tensor having a first number of discrete values within a first matrix of state features [mental process/step];
defining a reward function using a second helper function to generate a reward tensor having a second number of discrete values within a second matrix of state features [mental process/step]; and
using an MDP solver located on the UAS to generate the MDP policy based on the set of finite states, the set of actions, the probability tensor, and the reward tensor, wherein the MDP solver reduces the first number of discrete values of the probability tensor and the second number of discrete values of the reward tensor to a number of values for the set of actions for the UAS [mathematical concepts].
28. The mathematical concept(s) recited in claim 20 is/are: “generating a Markov Decision Process (MDP) policy” and “using an MDP solver located on the UAS to generate the MDP policy based on the set of finite states, the set of actions, the probability tensor, and the reward tensor, wherein the MDP solver reduces the first number of discrete values of the probability tensor and the second number of discrete values of the reward tensor to a number of values for the set of actions for the UAS.”
29. Under the broadest reasonable interpretation, the claim recites a mathematical concept – the above limitation(s) is/are steps in a mathematical concept such as mathematical relationships, mathematical formulas or equations, and mathematical calculations. If a claim, under its broadest reasonable interpretation, is directed towards a mathematical concept, then it falls within the Mathematical Concepts grouping of abstract ideas. In addition, as per MPEP § 2106.04(a)(2): “It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula." In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). See, e.g., SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 127 USPQ2d 1597, 1599 (Fed. Cir. 2018)” See MPEP § 2106.04(a)(2).
30. The mental process(es) recited in claim 20 is/are: defining a set of finite states …,” “defining a set of actions …,” “defining a transition function …,” “defining a reward function …”.
31. Under the broadest reasonable interpretation, these limitation(s) is/are process steps that cover mental processes including an observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper but for the recitation of a generic computer component. If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas. A person would readily be able to perform this process either mentally or with the assistance of pen and paper. See MPEP § 2106.04(a)(2). For example, “defining a set of finite states …,” “defining a set of actions …,” “defining a transition function …,” and “defining a reward function …” step(s) encompass(es) a user making observation, evaluation or judgement about the state of a UAS, could all be carried out in one’s mind. The same user looking at the data collected, could form a simple judgement and conclude whether to determine an action for the UAS based on the current state of the UAS. Accordingly, the claim recites at least one abstract idea.
32. As such, claim 20 recite an abstract idea of both a mental process and/or mathematical concept.
101 Analysis – Step 2A, Prong II
33. Regarding Prong II of the Step 2A analysis, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into a practical application. see MPEP 2106.04(II)(A)(2) and MPEP 2106.04(d)(2). It must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
34. For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
35. In regard to claim 20, the claim does not include any additional limitations.
36. Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception. see MPEP § 2106.05. Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
101 Analysis – Step 2B
37. Regarding Step 2B of the Revised Guidance, independent claim(s) 20 does/do not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application.
38. In regard to claim 20, as discussed above with respect to integration of the abstract idea into a practical application, there are no additional limitations.
39. Therefore, claim(s) 20 is/are ineligible under 35 USC §101.
Claim Rejections - 35 USC § 103
42. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
43. Claim(s) 1-3, 12 and 14
is/are rejected under 35 U.S.C. 103 as being unpatentable over Hudson et al. (US-20180227860-A1) in view of Bruni et al. (US-20130124076-A1) and further in view of Osipychev et al. (US-20230245575-A1).
In regard to claim 1
, Hudson discloses a route survivability autonomous system located within an unmanned aircraft system (UAS) comprising (Hudson, in at least [0004], discloses a UAS [i.e., an unmanned aircraft system (UAS)] is composed of three main parts, the unmanned air vehicle (UAV), unmanned control station (UCS) and support systems of the UAS (for pre-mission planning). A dynamic route planner [i.e., a route survivability autonomous system located within an unmanned aircraft system (UAS)] generates a new route, in real time, when there is a change in the operational environment, e.g. severe weather, pop-up threat, or a change of circumstances, e.g. an emergency, or a dynamic manoeuvre plan is generated to avoid an airborne obstacle. The aim is thus to maintain safety and the survivability of the aircraft [i.e., a route survivability] by determining a feasible route and/or manoeuvre in real time, while avoiding pop-up, static and dynamic obstacles. Examiner notes, the dynamic route planner is the route survivability autonomous system which has the responsibility for maneuvering the unmanned aircraft system to maintain safety and the survivability of the aircraft):
a subsystem configured to execute a state helper function to define a state having state features for the UAS, wherein the subsystem receives a threat location relative to the UAS and generates a threat severity level for a threat as a state feature for the state (Hudson, in at least Fig. 7, and [0097], discloses the emissions management module 606 [i.e., a subsystem configured to execute a state helper function to define a state having state features for the UAS] receives data regarding the threat(s), such as threat location [i.e., wherein the subsystem receives a threat location relative to the UAS], threat type, threat category and the EMCON level for a given threat [i.e., generates a threat severity level for a threat as a state feature for the state]);
(Hudson, in at least Fig. 7, and [0098-0100], discloses the emissions management module 606 also receives data regarding the movement of its own platform [i.e., configured to receive the state having the state features for the UAS], such as (from the mission management module 602) predetermined route plan or updated route plan (e.g. generated to avoid a pop-up threat), and (from the own ship mobility prediction engine 604) future prediction information about its own platform. An Emissions Plan is generated [i.e., to determine an action] based on dynamic movement of own aircraft and a dynamic pop-up threat(s));
a maneuver primitive (MP) helper to execute an action helper function based on the action determined (Hudson, in at least Figs. 2, 9, and [0063 & 0101], discloses the management function 41 is responsible for generating plan requests and providing attributes to the dynamic planner 40, evaluating new plans, selecting the best plan, requesting authorisation from the platform/pilot to execute the new plan [i.e., a maneuver primitive (MP) helper to execute an action helper function]. The emissions management module generates one or more Emissions Plans as the aircraft flies its pre-planned route. It uses the pre-determined route plan to generate the Emissions Plans. Thus, for each leg or segment along the route, there is an Emissions Plan in the direction of the threat 608 (segment 612) and in other directions (segment 614) [i.e., wherein the action helper function defines a guidance for the UAS based on the action]); and
Hudson is silent on using a threat probability model and a severity filter, and wherein the severity filter determines whether the threat is receding,
a Markov Decision Process (MDP) policy,
by the MDP policy,
a maneuver primitive supervisor (MPS) configured to receive the guidance based on the action from the MDP policy and to issue a command to the UAS, wherein the UAS performs the action based on the command.
However, Bruni teaches using a threat probability model and a severity filter, and wherein the severity filter determines whether the threat is receding (Bruni, in at least Fig. 4, and [0015 & 0051-0055], teaches determining a temporal plan comprises determining a first incremental instruction and a second incremental instruction from a Time-Dependent Markov Decision Process (TMDP). A hazard state estimation model [i.e., a threat probability model and a severity filter] determines a hazard state. The output of the hazard state estimation model is an estimate of the probability and severity of the hazards indicated by the sensor input. Generally, a decision-theoretic algorithm, such as but not limited to a TDMP, is used to produce the temporal plan. The TMDP model is used to capture the state uncertainty in the environment. A Markov model of the effectiveness of the automation in handling the hazards, given various levels of alert is an input to the planning algorithm. Its output is a representation of a system reaction, such as a plan for addressing the hazard. As the hazard level increases, different coloring of hazards on a graphic interface are displayed. Examiner notes, as mentioned above, the system determines whether the hazard level is increasing or not. To change or cancel the warning, the system necessarily is capable of determining whether the threat is receding in the same manner).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson in view of Bruni with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – determining and handling hazards – and use the hazard state estimation model of Bruni to output the probability and severity of the hazards indicated by the sensor input and to determine whether the threat level is increasing or decreasing and the combination would provide for increasing the safety and efficiency of air traffic (Bruni, see at least [0007]).
While Bruni teaches determining a temporal plan comprises determining a first incremental instruction and a second incremental instruction from a Time-Dependent Markov Decision Process (TMDP) (See at least [0015]), Hudson, as modified by Bruni, does not explicitly teach, in details, a Markov Decision Process (MDP) policy,
by the MDP policy,
a maneuver primitive supervisor (MPS) configured to receive the guidance based on the action from the MDP policy and to issue a command to the UAS, wherein the UAS performs the action based on the command.
However, Osipychev teaches a Markov Decision Process (MDP) policy (Osipychev, in at least Figs. 1-4, and [0028 & 0033-0044 & 0056], teaches an autonomous aerial robot, autonomous aerial vehicle or autonomous aircraft in the case of an autonomous robot that is also designed and configurable to fly. The MMS 210 is also configured to provide the aircraft 100 with conflict detection and avoidance capabilities. The MMS 210 is configured to receive observations of states of the aircraft 100 and the nearby obstacle 306 in the environment 304 of the aircraft as the aircraft travels the defined route 302. The MMS 210 is configured to output an indication of the collision avoidance trajectory for use in at least one of guidance, navigation or control of the aircraft. The states are applied to a reinforcement learning (RL) framework 406 that determines maneuvers of the aircraft 100 to avoid a conflict with the nearby obstacle 306, using a policy 408 trained using a surrogate model of the environment in which movements of the aircraft and the nearby obstacle are simulated. To solve the optimization problem as a Markov Decision Problem (MDP) system [i.e., a Markov Decision Process (MDP) policy], the problem is refactored into Markovian states s, transitions T(ś|s, α), and transition rewards R(ś|s, α). The state of the system (including both the agent and intruder) is fully observable, assume the perfect knowledge, and be enough to describe the Markovian state of the MDP system),
by the MDP policy (Osipychev, in at least [0056-0057], teaches to solve the optimization problem as a Markov Decision Problem (MDP) system [i.e., a Markov Decision Process (MDP) policy], the problem is refactored into Markovian states s, transitions T(ś|s, α), and transition rewards R(ś|s, α). The state of the system (including both the agent and intruder) is fully observable, assume the perfect knowledge, and be enough to describe the Markovian state of the MDP system. The optimization is set to find an optimal policy π*(s) [i.e., by the MDP policy] as a set of state-action mappings that maximizes an expected reward V(s))
a maneuver primitive supervisor (MPS) configured to receive the guidance based on the action from the MDP policy and to issue a command to the UAS, wherein the UAS performs the action based on the command (Osipychev, in at least [0031], teaches the aircraft 100 includes a vehicle management system (VMS) [i.e., a maneuver primitive supervisor (MPS)] 208 which is configured to accept maneuver commands such as waypoints and/or steering commands [i.e., configured to receive the guidance based on the action from the MDP policy and to issue a command to the UAS], and control the aircraft to follow those maneuver commands [i.e., wherein the UAS performs the action based on the command]. Examiner notes, for the VMS to accept the maneuver commands, the commands must be necessarily received).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as already modified by Bruni, in view of Osipychev with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and use Markov Decision Process (MDP) policy for optimization and determine the avoidance action based on the Markov Decision Process (MDP) policy and use commands for guiding the UAV and the combination would provide for including detect and avoid (DAA) capabilities to detect and avoid hazards in planning and execution of phases of flight, and to remain well clear of hazards by at least an appropriate separation distance (Osipychev, see at least [0004]).
In regard to claim 2
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, accordingly the rejection of claim 1 is incorporated.
Further, Osipychev teaches wherein the state features include a vehicle command to observe or to evade (Osipychev, in at least Fig. 7A. and [0061], teaches the method includes receiving observations of states of the aircraft and a nearby obstacle [i.e., wherein the state features include a vehicle command to observe] in an environment of the aircraft as the aircraft travels the defined route, as shown at block 702 of FIG. 7A. The method includes generating a collision avoidance trajectory [i.e., to evade] from the predicted states of the aircraft, as shown at block 710.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Osipychev with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – determining and handling hazards – and to observe the obstacle or to avoid the obstacle and the combination would provide for including detect and avoid (DAA) capabilities to detect and avoid hazards in planning and execution of phases of flight, and to remain well clear of hazards by at least an appropriate separation distance (Osipychev, see at least [0004]).
In regard to claim 3
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, wherein the state features include a previous action determine by the MDP policy (Hudson, in at least Figs. 2, 9, and [0063 & 0101], discloses the management function 41 is responsible for generating plan requests and providing attributes to the dynamic planner 40, evaluating new plans, selecting the best plan, requesting authorisation from the platform/pilot to execute the new plan. The emissions management module generates one or more Emissions Plans as the aircraft flies its pre-planned route. It uses the pre-determined route plan to generate the Emissions Plans. Thus, for each leg or segment along the route, there is an Emissions Plan in the direction of the threat 608 (segment 612) [i.e., wherein the state features include a previous action determine by the MDP policy] and in other directions (segment 614)).
In regard to claim 12
, Hudson discloses a method for operating an unmanned aircraft system (UAS) to evade a threat, the method comprising (Hudson, in at least Figs. 5-7, and [0022 & 0028], discloses a method of avoiding a conflict during a flight of an aircraft such as an unmanned aerial vehicles (UAVs)):
receiving a threat location of the threat at a state helper function of a subsystem within the UAS for a route survivability autonomous system for the UAS (Hudson, in at least Fig. 7, and [0097], discloses the emissions management module 606 [i.e., a state helper function of a subsystem within the UAS for a route survivability autonomous system for the UAS] receives data regarding the threat(s), such as threat location [i.e., receiving a threat location of the threat], threat type, threat category and the EMCON level for a given threat);
generating a state having state features using the state helper function (Hudson, in at least Fig. 7, and [0098], discloses the emissions management module 606 also receives data regarding the movement of its own platform [i.e., generating a state having state features using the state helper function], such as (from the mission management module 602) predetermined route plan or updated route plan (e.g. generated to avoid a pop-up threat), and (from the own ship mobility prediction engine 604) future prediction information about its own platform. Examiner notes, the emissions management module 606 is the state helper function);
determining an action for the UAS (Hudson, in at least [0100], discloses an Emissions Plan is generated [i.e., determining an action for the UAS] based on dynamic movement of own aircraft and a dynamic pop-up threat(s));
defining a guidance for the UAS using an action helper function within the UAS based on the action (Hudson, in at least Figs. 2, 9, and [0063 & 0101], discloses the management function 41 [i.e., using an action helper function based on the action] is responsible for generating plan requests and providing attributes to the dynamic planner 40, evaluating new plans, selecting the best plan, requesting authorisation from the platform/pilot to execute the new plan. The emissions management module generates one or more Emissions Plans as the aircraft flies its pre-planned route. It uses the pre-determined route plan to generate the Emissions Plans. Thus, for each leg or segment along the route, there is an Emissions Plan in the direction of the threat 608 (segment 612) and in other directions (segment 614) [i.e., defining a guidance for the UAS]);
Hudson is silent on using a threat probability model and a severity filter, and wherein the severity filter determines whether the threat is receding,
using a Markov Decision Process (MDP) policy,
based on the action from the MDP policy,
issuing a command to the UAS based on the guidance in response to the threat, and
performing the action by the UAS in response to the command.
However, Bruni teaches using a threat probability model and a severity filter, and wherein the severity filter determines whether the threat is receding (Bruni, in at least Fig. 4, and [0015 & 0051-0055], teaches determining a temporal plan comprises determining a first incremental instruction and a second incremental instruction from a Time-Dependent Markov Decision Process (TMDP). A hazard state estimation model [i.e., a threat probability model and a severity filter] determines a hazard state. The output of the hazard state estimation model is an estimate of the probability and severity of the hazards indicated by the sensor input. Generally, a decision-theoretic algorithm, such as but not limited to a TDMP, is used to produce the temporal plan. The TMDP model is used to capture the state uncertainty in the environment. A Markov model of the effectiveness of the automation in handling the hazards, given various levels of alert is an input to the planning algorithm. Its output is a representation of a system reaction, such as a plan for addressing the hazard. As the hazard level increases, different coloring of hazards on a graphic interface are displayed. Examiner notes, as mentioned above, the system determines whether the hazard level is increasing or not. To change or cancel the warning, the system necessarily is capable of determining whether the threat is receding in the same manner).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson in view of Bruni with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – determining and handling hazards – and use the hazard state estimation model of Bruni to output the probability and severity of the hazards indicated by the sensor input and to determine whether the threat level is increasing or decreasing and the combination would provide for increasing the safety and efficiency of air traffic (Bruni, see at least [0007]).
While Bruni teaches determining a temporal plan comprises determining a first incremental instruction and a second incremental instruction from a Time-Dependent Markov Decision Process (TMDP) (See at least [0015]), Hudson, as modified by Bruni, does not explicitly teach, in details, using a Markov Decision Process (MDP) policy,
based on the action from the MDP policy,
issuing a command to the UAS based on the guidance in response to the threat, and
performing the action by the UAS in response to the command.
However, Osipychev teaches using a Markov Decision Process (MDP) policy (Osipychev, in at least Figs. 1-4, and [0028 & 0033-0044 & 0056], teaches an autonomous aerial robot, autonomous aerial vehicle or autonomous aircraft in the case of an autonomous robot that is also designed and configurable to fly. The MMS 210 is also configured to provide the aircraft 100 with conflict detection and avoidance capabilities. The MMS 210 is configured to receive observations of states of the aircraft 100 and the nearby obstacle 306 in the environment 304 of the aircraft as the aircraft travels the defined route 302. The MMS 210 is configured to output an indication of the collision avoidance trajectory for use in at least one of guidance, navigation or control of the aircraft. The states are applied to a reinforcement learning (RL) framework 406 that determines maneuvers of the aircraft 100 to avoid a conflict with the nearby obstacle 306, using a policy 408 trained using a surrogate model of the environment in which movements of the aircraft and the nearby obstacle are simulated. To solve the optimization problem as a Markov Decision Problem (MDP) system [i.e., using a Markov Decision Process (MDP) policy], the problem is refactored into Markovian states s, transitions T(ś|s, α), and transition rewards R(ś|s, α). The state of the system (including both the agent and intruder) is fully observable, assume the perfect knowledge, and be enough to describe the Markovian state of the MDP system),
from the MDP policy (Osipychev, in at least [0056-0057], teaches to solve the optimization problem as a Markov Decision Problem (MDP) system [i.e., a Markov Decision Process (MDP) policy], the problem is refactored into Markovian states s, transitions T(ś|s, α), and transition rewards R(ś|s, α). The state of the system (including both the agent and intruder) is fully observable, assume the perfect knowledge, and be enough to describe the Markovian state of the MDP system. The optimization is set to find an optimal policy π*(s) [i.e., from the MDP policy] as a set of state-action mappings that maximizes an expected reward V(s)),
issuing a command to the UAS based on the guidance in response to the threat (Osipychev, in at least [0031], teaches the aircraft 100 includes a vehicle management system (VMS) 208 which is configured to accept maneuver commands such as waypoints and/or steering commands [i.e., issuing a command to the UAS based on the guidance in response to the threat], and control the aircraft to follow those maneuver commands. Examiner notes, for the VMS to accept the maneuver commands, the commands must be necessarily issued).
performing the action by the UAS in response to the command (Osipychev, in at least [0031], teaches the aircraft 100 includes a vehicle management system (VMS) 208 which is configured to accept maneuver commands such as waypoints and/or steering commands, and control the aircraft to follow those maneuver commands [i.e., performing the action by the UAS in response to the command]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as already modified by Bruni, in view of Osipychev with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and use Markov Decision Process (MDP) policy for optimization and determine the avoidance action based on the Markov Decision Process (MDP) policy and use commands for guiding the UAV and the combination would provide for including detect and avoid (DAA) capabilities to detect and avoid hazards in planning and execution of phases of flight, and to remain well clear of hazards by at least an appropriate separation distance (Osipychev, see at least [0004]).
In regard to claim 14
, Hudson, as modified by Osipychev, teaches the method of claim 12.
Claim 14 recites a method having substantially the same features of claim 2 above, therefore claim 14 is rejected for the same reasons as claim 2.
44. Claim(s) 4, and 15
is/are rejected under 35 U.S.C. 103 as being unpatentable over Hudson et al. (US-20180227860-A1) in view of Bruni et al. (US-20130124076-A1) and further in view of Osipychev et al. (US-20230245575-A1) and further in view of Cella et al. (US-20220187847-A1).
In regard to claim 4
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, accordingly the rejection of claim 1 is incorporated.
Hudson, as modified by Bruni and Osipychev, is silent on all limitations of the claim.
However, Cella teaches wherein the state features include a dead condition (Cella, in at least [0110], teaches if the sensor readings from a particular physical asset (e.g., a critical piece of manufacturing equipment) are indicative of a potentially critical situation (e.g., failure state, dangerous condition, or the like), then the analytics indicates a potentially critical situation [i.e., wherein the state features include a dead condition]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Cella with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and include a failure state or dead condition and the combination would provide for converting the data into insights and to translating the insights into well-informed decisions and timely execution of efficient operations (Cella, see at least [0006]).
In regard to claim 15
, Hudson, as modified by Bruni and Osipychev, teaches the method of claim 12.
Claim 15 recites a method having substantially the same features of claim 4 above, therefore claim 15 is rejected for the same reasons as claim 4.
45. Claim(s) 5-6, and 16-17
is/are rejected under 35 U.S.C. 103 as being unpatentable over Hudson et al. (US-20180227860-A1) in view of Bruni et al. (US-20130124076-A1) and further in view of Osipychev et al. (US-20230245575-A1) and further in view of Isorce et al. (US-20080172148-A1).
In regard to claim 5
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, accordingly the rejection of claim 1 is incorporated.
Hudson, as modified by Bruni and Osipychev, is silent on all limitations of the claim.
However, Isorce teaches wherein the guidance includes a follow route action for the UAS (Isorce, in at least Figs. 1-2, and [0044], teaches device 1, which secures low-altitude flight of the aircraft A, which is automatically guided along a low-altitude flight trajectory T0, which is followed by the aircraft A [i.e., wherein the guidance includes a follow route action for the UAS]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Isorce with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and include follow path or route action and the combination would provide for controlling an aircraft so as to increase its altitude and bring it to a security altitude, when said current real height becomes less than or equal to said current threshold value (Isorce, see at least [0036]).
In regard to claim 6
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, accordingly the rejection of claim 1 is incorporated.
Hudson, as modified by Bruni and Osipychev, is silent on all limitations of the claim.
However, Isorce teaches wherein the guidance includes a follow route at a low altitude action for the UAS (Isorce, in at least Figs. 1-2, and [0044], teaches device 1, which secures low-altitude flight of the aircraft A, which is automatically guided along a low-altitude flight trajectory T0, which is followed by the aircraft A [i.e., wherein the guidance includes a follow route at a low altitude action for the UAS]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Isorce with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and use an action for following a route at a low altitude and the combination would provide for controlling an aircraft so as to increase its altitude and bring it to a security altitude, when said current real height becomes less than or equal to said current threshold value (Isorce, see at least [0036]).
In regard to claim 16
, Hudson, as modified by Bruni and Osipychev, teaches the method of claim 12.
Claim 16 recites a method having substantially the same features of claim 5 above, therefore claim 16 is rejected for the same reasons as claim 5.
In regard to claim 17
, Hudson, as modified by Bruni and Osipychev, teaches the method of claim 12.
Claim 17 recites a method having substantially the same features of claim 6 above, therefore claim 17 is rejected for the same reasons as claim 6.
46. Claim(s) 7-8, and 18-19
is/are rejected under 35 U.S.C. 103 as being unpatentable over Hudson et al. (US-20180227860-A1) in view of Bruni et al. (US-20130124076-A1) and further in view of Osipychev et al. (US-20230245575-A1) and further in view of Summer et al. (US-20260003370-A1).
In regard to claim 7
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, accordingly the rejection of claim 1 is incorporated.
Hudson, as modified by Bruni and Osipychev, is silent on all limitations of the claim.
However, Summer teaches wherein the guidance includes a shallow turn action for the UAS (Summer, in at least [0008], teaches an aerial uncrewed vehicle has a minimum [i.e., wherein the guidance includes a shallow turn action for the UAS] and a maximum turn radius).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Summer with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and include a minimum turn action and the combination would provide for adjusting both the speed and direction of the travel trajectory (Summer, see at least [0008]).
In regard to claim 8
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, accordingly the rejection of claim 1 is incorporated.
Hudson, as modified by Bruni and Osipychev, is silent on all limitations of the claim.
However, Summer teaches wherein the guidance includes a maximum turn action for the UAS (Summer, in at least [0008], teaches an aerial uncrewed vehicle has a minimum and a maximum turn radius [i.e., wherein the guidance includes a maximum turn action for the UAS]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Summer with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and include a maximum turn action and the combination would provide for adjusting both the speed and direction of the travel trajectory (Summer, see at least [0008]).
In regard to claim 18
, Hudson, as modified by Bruni and Osipychev, teaches the method of claim 12.
Claim 18 recites a method having substantially the same features of claim 7 above, therefore claim 18 is rejected for the same reasons as claim 7.
In regard to claim 19
, Hudson, as modified by Bruni and Osipychev, teaches the method of claim 12.
Claim 19 recites a method having substantially the same features of claim 8 above, therefore claim 19 is rejected for the same reasons as claim 8.
47. Claim(s) 9
is/are rejected under 35 U.S.C. 103 as being unpatentable over Hudson et al. (US-20180227860-A1) in view of Bruni et al. (US-20130124076-A1) and further in view of Osipychev et al. (US-20230245575-A1) and further in view of Sinha et al. (US-20120286102-A1).
In regard to claim 9
, Hudson, as modified by Bruni and Osipychev, teaches the route survivability autonomous system of claim 1, accordingly the rejection of claim 1 is incorporated.
Hudson, as modified by Bruni and Osipychev, is silent on all limitations of the claim.
However, Sinha teaches wherein the command issued by the MPS is a lift vector and thrust command (Sinha, in at least [0117] and Claim 7, teaches when any aircraft is banked, the lift vector [i.e., wherein the command issued by the MPS is a lift vector] from the main wing shifts away from the vertical axis. The standard stick turn is translated into a set of commands including thrust [i.e., thrust command] differentiation of the motors on the aerial vehicle).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Sinha with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and include a lift vector and thrust command and the combination would provide for to utilized the UAV in a significant number of operating environments (Sinha, see at least [0005]).
48. Claim(s) 13
is/are rejected under 35 U.S.C. 103 as being unpatentable over Hudson et al. (US-20180227860-A1) in view of Bruni et al. (US-20130124076-A1) and further in view of Osipychev et al. (US-20230245575-A1) and further in view of Bush et al. (US-20220406200-A1).
In regard to claim 13
, Hudson, as modified by Bruni and Osipychev, teaches the method of claim 12, (Hudson, as modified by Bruni and Osipychev,, accordingly the rejection of claim 12 is incorporated.
Hudson, as modified by Bruni and Osipychev, is silent on all limitations of the claim.
However, Bush teaches further comprising determining a threat severity level using the state helper function for the state features (Bush, in at least Figs. 1-3, and [0090], teaches the first communication device 108 forwards the status message to the UTM device 106, which analyzes the information. The UTM device 106 filters the data based on relevancy or threat level [i.e., determining a threat severity level using the state helper function for the state features], which corresponds at least in part to proximity of the UAV 104 to a designated airspace that is reserved, at least temporarily, for MAV 102 traffic. Examiner notes, to filter threat level, the level must be necessarily determined which encompasses determining a threat severity level using the state helper function for the state features).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Hudson, as modified by Bruni and Osipychev, in view of Bush with a reasonable expectation of success, as all inventions are directed to the same field of endeavor – determining and handling hazards – and the combination would provide for regulating traffic flow and avoiding conflicts (Bush, see at least [0003]).
49. Claim(s) 20
is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US-20250085714-A1) in view of Malikopoulos (US-20090306866-A1).
In regard to claim 20
, Zhang discloses a method for generating a Markov Decision Process (MDP) policy for use in a route survivability autonomous system for an unmanned aircraft system (UAS), the method comprising (Zhang, in at least [0008 & 0034], discloses a method for UAV path planning. A shield module refers to a protection framework for preventing the algorithm from outputting an unsafe action [i.e., a method for … use in a route survivability autonomous system for an unmanned aircraft system (UAS)]. The framework consists of a finite-state reactive system, a state trace, a safety specification, a Markov decision process (MDP) [i.e., generating a Markov Decision Process (MDP) policy], a safety automaton and an observe function):
defining a set of finite states for the UAS (Zhang, in at least [0015], discloses the finite-state reactive system is M=(S,θ,L), in which S is a set of n states [i.e., defining a set of finite states for the UAS], i.e., S=[s.sub.t].sub.t=1.sup.n in which L represents an observed label and θ represents a state transition function.);
defining a set of actions for the UAS corresponding to the set of finite states (Zhang, in at least Function 1, and [0015], discloses an action a that is performed at the state s is defined as Function 1 [i.e., defining a set of actions for the UAS corresponding to the set of finite states]);
defining a transition function using a first helper function to generate a probability tensor (Zhang, in at least Function 2, and [0015], discloses the state transition function θ is expressed as Function 2 [i.e., defining a transition function using a first helper function to generate a probability tensor]. Examiner notes, Function 2 generates a probability tensor);
defining a reward function using a second helper function to generate a reward tensor (Zhang, in at least Function 11, and [0059], discloses the reward function Reward.sub.t is calculated from Function 11 [i.e., defining a reward function using a second helper function to generate a reward tensor]. Examiner notes, Function 11 generates a reward tensor); and
using an MDP solver located on the UAS to generate the MDP policy based on the set of finite states, the set of actions, the probability tensor, and the reward tensor, (Zhang, in at least [0034-0035] discloses the shield module is based on linear temporal logic (LTL). The shield module constructed in this invention refers to a protection framework for preventing the algorithm from outputting an unsafe action. The framework consists of a finite-state reactive system, a state trace, a safety specification, a Markov decision process (MDP) [i.e., using an MDP solver located on the UAS to generate the MDP policy], a safety automaton and an observe function. The shield module performs formal modeling of UAV path planning by using the finite-state reactive system M=(S,θ,L) [i.e., based on the set of finite states, the set of actions, the probability tensor, and the reward tensor] and state traces commonly used in the linear temporal logic, and constrains the state traces by the safety specification Φ.sup.safe. Then, the safety specification and a Markov process of UAV path planning are transformed into two safety automaton models respectively).
Zhang does not explicitly disclose a probability tensor having a first number of discrete values within a first matrix of state features,
having a second number of discrete values within a second matrix of state features,
wherein the MDP solver reduces the first number of discrete values of the probability tensor and the second number of discrete values of the reward tensor to a number of values for the set of actions for the UAS.
However, Malikopoulos teaches a probability tensor having a first number of discrete values within a first matrix of state features (Malikopoulos, in at least [0221], teaches the Markov decision process model consists of five elements: (a) decision epochs; (b) states; (c) actions; (d) the transition probability matrix [i.e., a probability tensor having a first number of discrete values within a first matrix of state features]; and (e) the transition cost (or reward) matrix. Examiner notes, a matrix by definition is a set of numbers arranged in rows and columns. The numbers arranged in the matrix are the discrete values within a first matrix of state features and the count of the numbers in the matrix is the first number),
a reward tensor having a second number of discrete values within a second matrix of state features (Malikopoulos, in at least [0221], teaches the Markov decision process model consists of five elements: (a) decision epochs; (b) states; (c) actions; (d) the transition probability matrix; and (e) the transition cost (or reward) matrix [i.e., a reward tensor having a second number of discrete values within a second matrix of state features]. Examiner notes, a matrix by definition is a set of numbers arranged in rows and columns. The numbers arranged in the matrix are the discrete values within a second matrix of state features and the count of the numbers in the matrix is the second number),
wherein the MDP solver reduces the first number of discrete values of the probability tensor and the second number of discrete values of the reward tensor to a number of values for the set of actions for the UAS (Malikopoulos, in at least [0221], teaches the Markov decision process model consists of five elements: (a) decision epochs; (b) states; (c) actions; (d) the transition probability matrix; and (e) the transition cost (or reward) matrix. In this framework, the decision maker is faced with the problem of influencing system behavior as it evolves over time, by making decisions (choosing actions). The objective of the decision maker is to select the course of action (control policy) which causes the system to perform optimally with respect to some predetermined optimization criterion. Decisions must anticipate costs (or rewards) associated with future system states-actions. This limitation, as best understood, was interpreted under its broadest reasonable interpretation consistent with the Applicant’s specification and the knowledge of one of ordinary skill in the art as using Markov decision process model for selecting the course of action (control policy) which causes the system to perform optimally with respect to some predetermined optimization criterion, which is the purpose of using Markov decision process model. This interpretation is aligned with a relevant paragraph [0022] of Applicant’s disclosure, which states: “The present disclosure is directed to an autonomy policy design method using a Markov Decision Process formulation focused on a decision scenario applicable to UAS flying at low altitude. The disclosed embodiments select a state and action set, and implemented state calculation and action execution helper functions. The disclosed embodiments implement a set of functions that calculate state transition probabilities and rewards for the particular problem, using a reduced order set of parameters to enable the user to tune the policy design to satisfy verification rules.” As such, this limitation is no more than applying Markov decision process model for determining the optimal course of action for the UAV).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the application to modify Zhang in view of Malikopoulos with a reasonable expectation of success, as both inventions are directed to the same field of endeavor – finding optimal course of action – and use Markov decision process model, including the transition probability matrix and the transition cost (or reward) matrix, to find an optimal course of action of the UAV and the combination would provide for generating control signals based on the set of optimum values to control operation of the system (Malikopoulos, see at least [0057]).
Conclusion
50. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Benosman et al. (US-20220121188-A1) teaches a controller for controlling a system having uncertainties in its dynamics subject to constraints on an operation of the
Joseph et al. (US-20240019581-A1) teaches using standard threat models with aircrafts.
51. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
52. A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
53. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Preston J Miller whose telephone number is (703)756-1582. The examiner can normally be reached Monday through Friday 7:30 AM - 4:30 PM EST.
54. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
55. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramya P Burgess can be reached at (571) 272-6011. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
56. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.J.M./Examiner, Art Unit 3661
/Tarek Elarabi/Primary Examiner, Art Unit 3661
Prosecution Insights