DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
Claims 1, 3-13, and 16-22 are pending and have been examined below.
Election/Restrictions
In light of the amendments to the claims, no claims are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected invention, there being no allowable generic or linking claim, since the current claims no longer require restriction. Election was made without traverse in the reply filed on 4/10/26.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims of the instant application are rejected on the ground of nonstatutory double patenting as being unpatentable over claims of U.S. Patent No. US12411500 in view of US20210125502. The claims are not patentably distinct from each other because of the similarity and obvious variants between the two claim sets. The correspondence between the claims of the instant application and those of US Patent document and supporting reference are listed in the table below.
Instant Application
US12411500
US20210125502
claim 1
claim 1
paragraphs 0018, 0082, 0132
claim 5
claim 3
n/a
claim 6
claim 4
n/a
claim 7
claim 5
n/a
claim 9
claim 7
n/a
claim 10
claim 8
n/a
claim 11
claim 1
paragraphs 0018, 0082, 0132
claim 12
claim 13
n/a
claim 16
claim 9
n/a
claim 17
claim 1
paragraphs 0018, 0082, 0132
claim 19
claim 13
n/a
claim 21
claim 1
paragraphs 0018, 0082, 0132
claim 22
claim 13
n/a
This modification of US12411500 in light of the secondary reference(s) is proper because the applied reference(s) is/are so related that the appearance of features shown in one would suggest the application of those features to the other. US12411500 and US20210125502 both disclose watercraft control systems. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention to modify the system in US12411500 to include the teaching of US20210125502 with a reasonable expectation of success in order to enhance the safety of the watercraft and the effectiveness of the collision avoidance by detecting maneuvers of the deterrents. See In re Rosen, 673 F.2d 388, 213 USPQ 347 (CCPA 1982); In re Carter, 673 F.2d 1378, 213 USPQ 625 (CCPA 1982), and In re Glavas, 230 F.2d 447, 109 USPQ 50 (CCPA 1956). Further, it is noted that case law has held that a designer skilled in the art is charged with knowledge of the related art; therefore, the combination of old elements, herein, would have been well within the level of ordinary skill. See In re Antle, 444 F.2d 1168,170 USPQ 285 (CCPA 1971) and In re Nalbandian, 661 F.2d 1214, 211 USPQ 782 (CCPA 1981).
Claim Rejections - 35 USC § 101
35 USC 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1, 3-10 and 16 is/are rejected under 35 USC 101 because the claimed invention is directed to an abstract idea, without significantly more. The rejected claim(s) is/are shown below with formatting and annotations that will be referred to throughout the analysis. Abstract ideas are in bold, followed by the abstract idea grouping in brackets. Additional elements are underlined, followed by the category of additional elements for which the additional element fails to integrate the abstract idea into a practical application, the category being listed in brackets.
1. A method of training a machine learning, ML algorithm to control a watercraft, wherein the watercraft is a submarine or a submersible submerged in water, the method implemented, at least in part, by a computer, comprising a processor and a memory, aboard the watercraft, the method comprising:
obtaining training data including respective sets of sensor signals, related to respective deterrents, and corresponding actions of a set of communicatively isolated watercraft, including a first watercraft [insignificant extra-solution activity – data gathering]; and
training the ML algorithm [mathematical concept] comprising determining relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof [mental process],
wherein determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof comprises detecting maneuvers of the respective deterrents [mental process].
3. The method according to claim 1, wherein determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof comprises recognizing patterns of the maneuvers of the respective deterrents [mental process].
4. The method according to claim 1, wherein determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof comprises classifying the respective deterrents [mental process].
5. The method according to claim 1, wherein obtaining the corresponding actions of the first watercraft comprises identifying actions performed by a human operator aboard the first watercraft [mental process].
6. The method according to claim 1, wherein obtaining the corresponding actions of the first watercraft comprises identifying remedial actions performed by a human operator aboard the first watercraft responsive to actions implemented by the ML algorithm [mental process].
7. The method according to claim 1, wherein the actions are selected from controlling: a buoyancy, a rudder, a control surface or plane, a thruster, a propeller, a propulsor, and/or a prime mover, of the watercraft [mental process (further narrows the mental process of determining relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof)].
8. The method according to claim 1, wherein the sets of sensor signals comprise SONAR signals [field of use and technological environment].
9. The method according to claim 1:
wherein the training data include respective policies and corresponding trajectories of the set of watercraft, wherein each policy relates to navigating a watercraft of the set thereof in the water away from a deterrent and wherein each corresponding trajectory comprises a series of states in a state space of the watercraft [mental process]; and
wherein training the ML algorithm comprising determining relationships between the respective policies and corresponding trajectories of the watercraft of the set thereof based on respective results of comparing the trajectories and the deterrents [mental process].
10. The method according to claim 9, wherein the ML algorithm comprises and/or is a reinforcement learning, RL, agent, and wherein training the ML algorithm comprises training the agent [mathematical concept], the training comprising:
(a) actioning, by the agent, a watercraft of the set thereof according to a respective policy, wherein the policy is of an action space of the agent, comprising navigating the watercraft of the set thereof away from a deterrent, thereby defining a corresponding trajectory comprising a series of states in a state space of the watercraft and thereby obtaining respective training data [mental process];
(b) determining a relationship between the policy and the trajectory based on a result of comparing the trajectory and the deterrent and updating the policy based on the result [mental process]; and
(c) repeating (a) and (b) for the set of watercraft, using the updated policy [mental process].
16. The method according to claim 7, wherein the control surface or plane is a bow plane, a sail plane, or a stern plane [mental process (further narrows the mental process of determining relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof)].
Step 1
The claims are directed to a process, machine, manufacture or composition of matter. The analysis proceeds to step 2A, prong I.
Step 2A, Prong I
The claims are analyzed to determine whether they recite subject matter that falls within one of the follow groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes. See MPEP 2106(A)(11)(1) and MPEP 2106.04(a)-(c).
Examiner asserts that the foregoing bolded limitation(s) constitute(s) a mathematical concept, a certain method of organizing human activity, and/or a mental process because under the broadest reasonable interpretation, the limitation(s) can be performed in the human mind, or by a human using a pen and paper. Accordingly, the claim recites at least one abstract idea, and the analysis proceeds to step 2A, prong II.
Step 2A, Prong II
The claims are analyzed to determine whether the claims, as a whole, integrate the abstract idea(s) into a practical application. See MPEP 2106.04(11)(A)(2) and MPEP 2106.04(d)(2). It must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional element(s) merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.” See MPEP 2106.05f-h.
The additional element(s) of the claim at issue do not integrate the abstract idea into a practical application. Further, looking at the additional element(s) as an ordered combination or as a whole, the additional element(s) add nothing that is not already present when looking at the element(s) individually. For instance, there is no indication that the additional element(s), when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception. See MPEP 2106.05. The analysis proceeds to step 2B.
Step 2B
Step 2B requires that any additional element(s) determined to be insignificant extra solution activity in Step 2A must be re-evaluated in Step 2B to determine if the additional element(s) are more than what is well-understood, routine, and conventional in the field, which would result in the claim amounting to an inventive concept (in other words, "significantly more" than the abstract idea). Such element(s) in the claims being considered is/are listed below, each followed by the support showing that the element is well-understood, routine, and conventional in the field (see MPEP 2106.05dII for more details):
obtaining training data including respective sets of sensor signals, related to respective deterrents, and corresponding actions of a set of communicatively isolated watercraft, including a first watercraft – US12090273 (col. 2 lines 55-64 The generation of the machine learning model involves receiving or collecting training data in the form of predetermined datasets to train at least one neural network. A form of this neural network could be an edge-implemented deep neural net-based object detector which is well known in the art Other forms of machine learning other than neural networks can be substituted, as would be well known to a person of skill in the art.)
Thus, the claims fail to recite anything sufficient to amount to significantly more than the judicial exception.
Conclusion
Based on the analysis above, Examiner determines that claims 1, 3-10 and 16 do not qualify as eligible subject matter, and the claims are rejected under 35 USC 101.
Examiner notes that claim(s) 11 recite(s) additional elements that do result in the claims integrating the abstract idea into a practical application of the exception and amounting to an inventive concept (aka "significantly more"). Amending the independent claim(s) to include the subject matter of claim(s) 11 and any intervening claims, if applicable, would be sufficient to overcome the rejection under 35 USC 101.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 USC 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 4, 5, 7, 11-13 and 17-22 are rejected under 35 USC 102 as being anticipated by US20210125502 (“Mansor”).
Claim 1
Mansor discloses a method of training a machine learning, ML algorithm to control a watercraft, wherein the watercraft is a submarine or a submersible submerged in water, the method implemented, at least in part, by a computer, comprising a processor and a memory, aboard the watercraft (0095 model is trained, 0089 machine learnt model, 0042 The vessels may be any size and type and may include, for example, ocean going cargo vessels, passenger vessels, fishing trawlers or tug boats amongst others., 0146 processor, 0147 memory), the method comprising:
obtaining training data including respective sets of sensor signals, related to respective deterrents, and corresponding actions of a set of communicatively isolated watercraft, including a first watercraft (0049 The vessel data may be any available data which allows the collision risk to be assessed but will generally include a current (or relevant recent) position, speed and heading. Such information may be provided by radar, AIS, GPS or a similar system or other means of communication, 0048 The inputs 102 to the system 100 may include real-time vessel data from the two or more vessels and vessel manoeuvrability for the two or more vessels., 0029 comparing the real time data of the two or more vessels to historical real-time data from a plurality of historical vessel journeys; assessing which of the historical vessel journeys include portions which correspond to the determined collision risk; determining which of the portions of historical vessel journeys included risk reducing manoeuvres which resulted in reduced collision risks; and, outputting the risk reducing manoeuvres as the one or more collision avoidance manoeuvres for the one or more vessels., 0096 The historical data described below is AIS data, but it will be appreciated that other types or sources of historical data may be applicable. The historical data may include multiple types or sources of data. For example, the historical data may include incident reports capturing previous marine incidents not represented in the chosen primary source of historical data.); and
training the ML algorithm comprising determining relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof (0026 determining one or more collision avoidance manoeuvres on the basis of historical navigational data which corresponds to the real-time data;, 0060 The offline aspect relates in part to the training of a model which may be used to determine a collision risk and a collision avoidance strategy. The collision avoidance strategy may include a collision avoidance calculation for the own vessel and a consideration of one or more target vessels. Thus, the collision avoidance calculation may include one or more models. The models may be so-called machine learnt, or machine trained models., 0095 Thus, a model is trained using historical vessel navigation data prior to being used to provide a solution for a real-time scenario. Generally, the historical data is used to identify prior navigations which have a significant reduction in collision risk, and use those journeys to identify suitable navigation options for reducing collision risks. Providing a model with this information allows real-time scenarios to be mapped against prior collision risk behaviour and thus good practice., 0123 The objective of the optimisation is to find the range of manoeuvres which fits the output of the collision avoidance model, and minimises the deviation from the own planned route and existing sea lanes, and minimises the overall traffic collision risk, taking into account the predicted response from the other vessels in the vicinity to the own vessel manoeuvre. The optimisation step may include multiple separate algorithms., 0097 A collision avoidance calculation 600 is shown in FIG. 6. The collision avoidance calculation 600 may include one or more models which map historical data in which a collision risk is detected and avoided to real-time data., 0049, 0132),
wherein determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof comprises detecting maneuvers of the respective deterrents (0018 The one or more vessels may include an own vessel and one or more target vessels. The method may further comprise: determining one or more counter-navigation for the target vessels in response to the one or more collision avoidance manoeuvres., 0080 The historical data may relate to one or more of a: historical vessel data including one or more of a vessel type, speed, heading, rate of turning, route, speed, and historical sea state., 0132).
Claim 4
Mansor discloses:
wherein determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof comprises classifying the respective deterrents (0049 The vessel data may be any available data which allows the collision risk to be assessed but will generally include a current (or relevant recent) position, speed and heading. Such information may be provided by radar, AIS, GPS or a similar system or other means of communication (e.g. transmitted via a vessel communications platform). Vessel data may be known from vessel-based systems and/or general knowledge of the vessel or vessel type.).
Claim 5
Mansor discloses:
wherein obtaining the corresponding actions of the first watercraft comprises identifying actions performed by a human operator aboard the first watercraft (0132 In use, the trained model may be used by a vessel or remote/on-board operator to input the real-time data as described above. The model effectively mapping the real-time data input via the model parameters to provide one or more outputs. The one or more outputs may be time separated manoeuvres., 0080 The historical data may relate to one or more of a: historical vessel data including one or more of a vessel type, speed, heading, rate of turning, route, speed, and historical sea state. Thus, in a scenario where a target vessel manoeuvrability may not be determined from the available vessel data but its type identified, by, for example, observation, historical data may be used to determine the manoeuvrability on the basis of the identified vessel type as it appears in the historical data., 0064 The remote operator 110 shown on the left hand side of FIG. 2 may be shore based and may act on behalf of the vessel and or other vessels in the area. Alternatively, the operator 110 may be on-board a vessel).
Claim 7
Mansor discloses:
wherein the actions are selected from controlling: a buoyancy, a rudder, a control surface or plane, a thruster, a propeller, a propulsor, and/or a prime mover, of the watercraft (0071 The historical vessel data 402 may include vessel characteristics such as the type of hull, rudder, engines and manner of control (such as autopilot) and may be provided with contemporaneous historical sea-state data such that the vessel manoeuvrability for a particular vessel characteristic set operating in a particular sea-state or perhaps a geographical location may be mapped.).
Claim 11
Mansor discloses:
a method of controlling a communicatively isolated watercraft (0002 system and method may be utilised by manned, remotely operated or autonomously controlled marine vessels.), wherein the watercraft is a submarine or a submersible submerged in water (0042 The vessels may be any size and type and may include, for example, ocean going cargo vessels, passenger vessels, fishing trawlers or tug boats amongst others.), the method implemented, at least in part, by a computer, comprising a processor and a memory, aboard the watercraft (0095 model is trained, 0089 machine learnt model, 0042 The vessels may be any size and type and may include, for example, ocean going cargo vessels, passenger vessels, fishing trawlers or tug boats amongst others., 0146 processor, 0147 memory), the method comprising:
controlling, by a trained machine learning, ML, algorithm according to claim 1, the watercraft, the controlling comprising navigating the watercraft away from a deterrent (0095 model is trained, 0089 machine learnt model, 0042 The vessels may be any size and type and may include, for example, ocean going cargo vessels, passenger vessels, fishing trawlers or tug boats amongst others., 0146 processor, 0147 memory, 0002 system and method may be utilised by manned, remotely operated or autonomously controlled marine vessels., claim 1: collision avoidance, 0010, 0013).
Claim 12
Mansor discloses:
wherein the watercraft is an autonomous and/or unmanned watercraft (0042 In one example, there may be two ships within a predetermined distance of one another as provided by AIS or radar data. One or more of the vessels may be autonomous having a relatively small number of on-board crew, potentially zero, and possibly one or more shore-based remote operators.).
Claim 13
Mansor discloses:
obtaining a set of sensor signals, related to the deterrent (0049 The vessel data may be any available data which allows the collision risk to be assessed but will generally include a current (or relevant recent) position, speed and heading. Such information may be provided by radar, AIS, GPS or a similar system or other means of communication, 0048 The inputs 102 to the system 100 may include real-time vessel data from the two or more vessels and vessel manoeuvrability for the two or more vessels., 0029 comparing the real time data of the two or more vessels to historical real-time data from a plurality of historical vessel journeys; assessing which of the historical vessel journeys include portions which correspond to the determined collision risk; determining which of the portions of historical vessel journeys included risk reducing manoeuvres which resulted in reduced collision risks; and, outputting the risk reducing manoeuvres as the one or more collision avoidance manoeuvres for the one or more vessels., 0096 The historical data described below is AIS data, but it will be appreciated that other types or sources of historical data may be applicable. The historical data may include multiple types or sources of data. For example, the historical data may include incident reports capturing previous marine incidents not represented in the chosen primary source of historical data.).
Claim(s) 17
Claim(s) 17 recite(s) subject matter similar to that/those of claim(s) 11 and is/are rejected under the same grounds.
Claim 18
Mansor discloses:
wherein navigating the watercraft away from the deterrent is according to a policy (0075 As will be appreciated, the evasive action required in light of a particular level of collision risk will vary depending on the circumstances. In some cases, a minor evasive manoeuvre in line with standard COLREGS navigation may be required, in which case an early manoeuvre may have minimal impact on a voyage overall. However, if action is delayed the collision risk may increase and a more severe evasive manoeuvre may be required. Such evasive actions may include a change in trajectory of the own vessel by altering one or more of a change of direction or a change in speed, perhaps to a dead stop, or by hailing the target vessel if they are required to undertake a corresponding manoeuvre.).
Claim(s) 19, 20, 21 and 22
Claim(s) 19, 20, 21 and 22 recite(s) subject matter similar to that/those of claim(s) 12, 13, 17 and 12, respectively, and is/are rejected under the same grounds.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 3 is rejected under 35 USC 103 as being unpatentable over Mansor in view of US20210358309 (“Watanabe”).
Claim 3
Mansor fails to disclose wherein determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof comprises recognizing patterns of the maneuvers of the respective deterrents. However, Mansor does disclose determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof (0026, 0060, 0095). Furthermore, Watanabe teaches a system of controlling a watercraft based on past data (0143), including:
wherein determining the relationships between the respective sets of sensor signals and the corresponding actions of the watercraft of the set thereof comprises recognizing patterns of the maneuvers of the respective deterrents (0149 The collision risk calculation device 10 specifies a time range before a maximum risk value from the calculated risk values. The collision risk calculation device 10 extracts, from the track data, an action pattern of one or both of the first vessel and the second vessel in the time range. The collision risk calculation device 10 calculates a degree of the action pattern of one or both of the first vessel and the second vessel on the basis of the action pattern. The collision risk calculation device 10 corrects the risk value on the basis of the degree of the action pattern and a weight set for each action pattern.).
Mansor and Watanabe both disclose systems of controlling watercraft in response to deterrents. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention to modify the system in Mansor to include the teaching of Watanbe with a reasonable expectation of success in order to enhance the likelihood of control of the watercraft to result in safe maneuvering.
Claim 6 is rejected under 35 USC 103 as being unpatentable over Mansor in view of US20060278152 (“Nickerson”).
Claim 6
Mansor fails to disclose wherein obtaining the corresponding actions of the first watercraft comprises identifying remedial actions performed by a human operator aboard the first watercraft responsive to actions implemented by the ML algorithm. However, Mansor does disclose obtaining corresponding actions of the first watercraft (0049). Furthermore, Nickerson teaches a system of controlling a watercraft (abstract), including:
wherein obtaining the corresponding actions of the first watercraft comprises identifying remedial actions performed by a human operator aboard the first watercraft responsive to actions implemented by an autonomous control system (0001 invention relates to a device installed on a boat that is integrated with a boat autopilot and allows a boat operator to automatically override an autopilot function and turn the boat at a rate and direction that is approximately proportional to the rate and direction of a steering device such as a steering wheel. More particularly, for example, if the operator of the boat turns the steering wheel at a consistent rate of 90 degrees per second, the turn rate of the boat will be a consistent 10 degrees per second regardless of the speed of the boat. When the steering wheel motion is stopped, a heading reference is sent to an autopilot controller and the autopilot will maintain the course that was set when the steering wheel motion is stopped.). Examiner notes that while the system in Mansor discloses an autonomous control system influenced by the actions implemented by the ML algorithm, and Nickerson teaches identifying remedial actions performed by a human operator aboard the first watercraft responsive to actions implemented by an autonomous control system, one of ordinary skill in the art would have acknowledged that the combination of the prior art would have resulted in the invention as claimed wherein obtaining the corresponding actions of the first watercraft comprises identifying remedial actions performed by a human operator aboard the first watercraft responsive to actions implemented by the ML algorithm.
Mansor and Nickerson both disclose systems of controlling a watercraft. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention to modify the system in Mansor to include the teaching of Nickerson with a reasonable expectation of success in order to improve safety of watercraft by ensuring that human may take control over the watercraft in the case that the actions implemented by the ML algorithm are deemed to be unsafe or suboptimal.
Claim 8 is rejected under 35 USC 103 as being unpatentable over Mansor in view of US20200012283 (“Nguyen”).
Claim 8
Mansor fails to disclose wherein the sets of sensor signals comprise SONAR signals. However, Mansor does disclose various sets of known watercraft sensor signals (0049 The vessel data may be any available data which allows the collision risk to be assessed but will generally include a current (or relevant recent) position, speed and heading. Such information may be provided by radar, AIS, GPS or a similar system or other means of communication (e.g. transmitted via a vessel communications platform). Vessel data may be known from vessel-based systems and/or general knowledge of the vessel or vessel type.). Furthermore, Nguyen teaches a system of controlling a watercraft based on machine learning and deterrents (0002 invention relates to the field of autonomous maritime vessel navigational safety and security utilizing imaging and audio sensors, computer vision, and machine learning. Embodiments of the present invention provide for a piloted and/or non-piloted marine vessel facilitating and supporting autonomous onboard capabilities for safety (i.e. collision avoidance, man-overboard, etc.) and security (i.e. intrusion, illegal boarding, etc.).), including:
wherein the sets of sensor signals comprise SONAR signals (0061 invention relates to maritime hazard mitigations and methods thereof that provide for a contextual understanding of a marine environment to identify situations of collisions, man-overboard, intrusion and taking appropriate action based on context. This includes imaging (conventional camera, ToF camera, depth camera, thermal cameras, radar, lidar) and audio (microphone, sonar, sonic) sensors, compute device to build environmental understand, recognition, and compute optimal route navigation, controller to manage heading, controller to handle propulsion, display for latest marine information and navigation data, speakers to alert crew, horn to signal to other vessels.).
Mansor and Nguyen both disclose systems of controlling watercraft for collision avoidance based on machine learning. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention to modify the system in Mansor to include the teaching of Nguyen with a reasonable expectation of success in order to additionally capture information regarding deterrents underwater that the other sensors of Mansor may not be able to capture by conventional means.
Claim 9 is rejected under 35 USC 103 as being unpatentable over Mansor in view of US20160299507 (“Shah”).
Claim 9
Mansor discloses:
wherein the training data include respective policies and corresponding trajectories of the set of watercraft, wherein each policy relates to navigating a watercraft of the set thereof in the water away from a deterrent (0089 In such a case, a manoeuvre may be selected from a rule-based decision system that represents, for example, COLREGS. The rule-based decision may include the use of a look-up table or decision tree for example. In this case, there is no need to undertake the collision avoidance calculation 316 shown in FIG. 3. Typically, the rule-based decision system will be used to ensure there is a compliant solution which could be executed for cases where the machine learnt model solutions are not suitable, for example, if they do not converge., 0090 COLREGS provide navigational rules to help avoid collisions but are only basic rules which require assessment and application to a given scenario by a vessel operator. For example, in a head-on encounter, COLREGS Rules 14 and 16 which require a vessel to alter course to starboard and to do so early and substantially may apply. However, no heading change is specified, nor a time interval in which action should be taken. These decisions are made locally by the mariner. Hence, the low threshold may provide upper and lower bounds for instructed actions which are interpreted using the rule based system. A “back-up” manoeuvre may also be constructed using the rule based system. For example, the heading change may be taken as a medial value between the maximum and minimum threshold.); and
wherein training the ML algorithm comprising determining relationships between the respective policies and corresponding trajectories of the watercraft of the set thereof based on respective results of comparing the trajectories and the deterrents (0089 In such a case, a manoeuvre may be selected from a rule-based decision system that represents, for example, COLREGS. The rule-based decision may include the use of a look-up table or decision tree for example. In this case, there is no need to undertake the collision avoidance calculation 316 shown in FIG. 3. Typically, the rule-based decision system will be used to ensure there is a compliant solution which could be executed for cases where the machine learnt model solutions are not suitable, for example, if they do not converge., 0090 COLREGS provide navigational rules to help avoid collisions but are only basic rules which require assessment and application to a given scenario by a vessel operator. For example, in a head-on encounter, COLREGS Rules 14 and 16 which require a vessel to alter course to starboard and to do so early and substantially may apply. However, no heading change is specified, nor a time interval in which action should be taken. These decisions are made locally by the mariner. Hence, the low threshold may provide upper and lower bounds for instructed actions which are interpreted using the rule based system. A “back-up” manoeuvre may also be constructed using the rule based system. For example, the heading change may be taken as a medial value between the maximum and minimum threshold., 0122 The collision avoidance model 602, target vessel model 604 and COLREGS compliance manoeuvre calculation 606 may be subject to a de-normalisation step 640. The objective of the de-normalisation is to adjust the output of the collision avoidance model to scale it to the real-time scenario., 0097 A collision avoidance calculation 600 is shown in FIG. 6. The collision avoidance calculation 600 may include one or more models which map historical data in which a collision risk is detected and avoided to real-time data. The models may include a collision avoidance model 602 which provides one or more own vessel manoeuvres which potentially reduce the collision risk for the own vessel. Another model may be a target vessel model 604 which models the potential manoeuvres of target vessels in response to a proposed own vessel manoeuvre. A COLREGS model 606 may be provided to determine whether a proposed manoeuvre is COLREGS compliant.).
Mansor fails to disclose wherein each corresponding trajectory comprises a series of states in a state space of the watercraft. However, Mansor does disclose trajectories of the watercraft (0093). Furthermore, Shah teaches a system of watercraft collision avoidance control (abstract), including:
wherein each corresponding trajectory comprises a series of states in a state space of the watercraft (abstract, 0042 After the scaling at 312, the trajectory planning algorithm can evaluate a cost function at each state of the state space. The cost function can be based on at least predicted movement of obstacles in the travel space as they respond to the control action of the surface vehicle. Based on the cost function evaluation, a control action primitive that is best suited for that particular time step and state is selected at 316. If a prior control action primitive was selected in a previous step, then it can be combined with the current selection at 318. Repeating the cycle from 310 to 320 can yield a fully planned trajectory from current state to goal state that is composed of individual control action primitives from each time step coupled together).
Mansor and Shah both disclose systems of providing collision avoidance control for watercraft. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention to modify the system in Mansor to include the teaching of Shah with a reasonable expectation of success in order to allow for handling multiple inputs and outputs in the system, thus improving accuracy and robustness of the control system for which state space representation is known to provide.
Claim 10 is rejected under 35 USC 103 as being unpatentable over Mansor in view of Shah, in further view of US20220189312 (“Ma”).
Claim 10
Mansor fails to disclose: wherein the ML algorithm comprises and/or is a reinforcement learning, RL, agent, and wherein training the ML algorithm comprises training the agent, the training comprising: (a) actioning, by the agent, a watercraft of the set thereof according to a respective policy, wherein the policy is of an action space of the agent, comprising navigating the watercraft of the set thereof away from a deterrent, thereby defining a corresponding trajectory comprising a series of states in a state space of the watercraft and thereby obtaining respective training data; (b) determining a relationship between the policy and the trajectory based on a result of comparing the trajectory and the deterrent and updating the policy based on the result; and (c) repeating (a) and (b) for the set of watercraft, using the updated policy. However, Mansor does disclose navigating the watercraft of the set thereof away from a deterrent (0026, 0060, 0095). Furthermore, Ma teaches:
wherein the ML algorithm comprises and/or is a reinforcement learning, RL, agent, and wherein training the ML algorithm comprises training the agent (0013 Integrating LSTM neural network and deep reinforcement learning principles to build a collision avoidance training model for a swarm of unmanned surface vehicles;), the training comprising:
(a) actioning, by the agent, a watercraft of the set thereof according to a respective policy, wherein the policy is of an action space of the agent, comprising navigating the watercraft of the set thereof away from a deterrent, thereby defining a corresponding trajectory comprising a series of states in a state space of the watercraft and thereby obtaining respective training data (0034 the Cartesian two-dimensional coordinate system and the polar coordinate system are merged to characterize the relative orientation and movement relationship of obstacles in the vehicle coordinate system. Among them, the motion attributes of USV.sub.i in the global coordinate system include: course C.sub.i, velocity V.sub.i, rudder angle δ.sub.i, and position (P.sub.x_i, P.sub.y_i), the motion attributes of USV.sub.i in the vehicle coordinate system are: course c.sub.j, velocity v.sub.j, vehicle-side angle θ.sub.j, rudder angle ψ.sub.j, and location (X.sub.j, Y.sub.j)., 0017 At the same time, a reward and punishment function is designed to judge the collision avoidance effect of USV, which can judge the collision avoidance effect of USV in the current state and feed it back to the collision avoidance training model., 0019 principle of deep reinforcement learning enables the USV to interact with the training environment and learn collision avoidance actions, independently update the network parameters, and finally realize the safe avoidance of the USV in the swarm of unmanned surface vehicles collision avoidance environment., 0044);
(b) determining a relationship between the policy and the trajectory based on a result of comparing the trajectory and the deterrent and updating the policy based on the result (0059 After constructing the collision avoidance model for a swarm of unmanned surface vehicles, it is necessary to build a simulation environment that can simulate the collision avoidance of a swarm of unmanned surface vehicles. Through the interaction between the simulation environment and the model, a large number of training samples are generated and the model network parameters are gradually updated, as shown in FIG. 7., claim 1); and
(c) repeating (a) and (b) for the set of watercraft, using the updated policy (0059 After constructing the collision avoidance model for a swarm of unmanned surface vehicles, it is necessary to build a simulation environment that can simulate the collision avoidance of a swarm of unmanned surface vehicles. Through the interaction between the simulation environment and the model, a large number of training samples are generated and the model network parameters are gradually updated, as shown in FIG. 7.... secondly, setting the obstacles in the simulation environment to be USVs, which are used to generate training samples, that is, each USV is determined by the model to avoid collisions, and the generated (s.sub.t,a.sub.t,r.sub.t,s.sub.t+1) will be stored in the experience pool of the model for neural network training parameters; finally, the USV will continue to interact and iterate with the training model in the simulation environment until all unmanned surface vehicles can safely drive past and clear to complete collision avoidance., claim 1).
Mansor and Ma both disclose collision avoidance systems for watercraft based on machine learning. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention to modify the system in Mansor to include the teaching of Ma with a reasonable expectation of success in order to benefit from the advantages of reinforcement learning, specifically the ability to adapt dynamically to the environment in real-time without the need for pre-labeled data.
Claim 16 is rejected under 35 USC 103 as being unpatentable over Mansor in view of US11608149 (“Seeley”).
Claim 16
Mansor fails to disclose wherein the control surface or plane is a bow plane, a sail plane, or a stern plane. However, Mansor does disclose control of a vehicle component (0071). Furthermore, Seeley teaches control of an autonomous watercraft (Fig. 4a, abstract), including:
wherein the control surface or plane is a bow plane, a sail plane, or a stern plane (col. 6 line 65 – col. 7 line 8 FIG. 1D shows that in addition to discrete modules 108, various control surfaces 120, 122, and 124, propulsion mechanisms 130 and other external attachments may be attached to configure vehicle 100. In FIG. 1D, control surface 122 comprises a sail plane and control surface 120 comprises a stabilizer. Control surfaces 122 and 120, as is known to those of skill in the art, orient the vehicle in pitch, roll and yaw. Different types of control surfaces beyond those shown in FIG. 1D, including but not limited to, rudders, elevators, bow planes, and canards may also be attached or detached to reconfigure vehicle 100 as desired.).
Mansor and Seeley both disclose systems of controlling a watercraft. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention to modify the system in Mansor to include the teaching of Seeley with a reasonable expectation of success in order to ensure precise and safe control of the watercraft in the water.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Examiner KRISHNAN RAMESH whose telephone number is (571)272-6407. The examiner can normally be reached Monday-Friday 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn, can be reached at (571)272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISHNAN RAMESH/
Primary Examiner, Art Unit 3663
Prosecution Insights