DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/26/2026 has been entered.
Response to Arguments
Applicant’s arguments, filed 2/26/2026, have been fully considered and the examiner’s responses are given below.
The 35 U.S.C. 103 rejections are withdrawn, however new grounds are presented below.
Applicant’s amendments to the independent claims alter the scope of the claims, therefore new prior art has been applied and applicant’s arguments are moot.
Applicant argues that Vanhakartano does not teach the viewpoint position and line-of-sight direction correspond to the real-time position and attitude of a head-mounted display (HMD). Examiner respectfully disagrees. Vanhakartano teaches the display 107 can be implemented in smartglasses (with augmented reality) (Vanhakartano – Paragraph 0033). Augmented reality means the viewpoint position and line-of-sight direction corresponds to the position and attitude of the HMD.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Vanhakartano (US 20200278433 A1, cited in a previous office action) in view of Kim (US 20180164801 A1, cited in a previous office action) and Bradski (US 20190094981 A1).
Regarding claim 1, Vanhakartano discloses an augmented reality (AR) vessel maneuvering system, comprising (Paragraphs 0025-0036);
processing circuitry configured to (Paragraphs 0021-0025, 0035-0036);
generate an image including a vessel object representing a vessel in a region (Paragraphs 0025-0033, 0045, 0049-0051, Fig. 8; Vessel object is mapped to virtual model 108);
corresponding to a viewpoint position and a line-of-sight direction (Paragraphs 0025-0034, 0049-0051, Fig. 11);
superimpose and display the image including the vessel object on an outside scene of the region (Paragraphs 0045, 0049-0051, 0076, Fig. 8);
corresponding to the viewpoint position and the line-of-sight direction (Paragraphs 0025-0034, 0049-0051);
the vessel object (Paragraphs 0025-0033, 0045, 0049-0051, Fig. 8);
the vessel (Paragraph 0073).
Vanhakartano does not specifically state detect an operation on the object displayed in the image; and execute a navigation operation to navigate the object based on the operation on the virtual object.
However, Kim teaches detect an operation on the object displayed in the image (Paragraph 0135);
execute a navigation operation to navigate the object based on the operation on the virtual object (Paragraph 0135-0136).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with detecting an operation on the object displayed in the image and navigating the object based on the operation on the virtual object of Kim with a reasonable expectation of success. One of ordinary skill in the art would understand that vehicles such as the vessel in Vanhakartano and the UAV in Kim need to stably maneuver without colliding into obstacles. Allowing an electronic device to control the vehicle by dragging a virtual object to a position and attitude is simple and allows the vehicle to be safely controlled within specified ranges. One would have been motivated to combine Vanhakartano with Kim as this achieves stable operation of the vehicle. As stated in Vanhakartano, “the mariner 152B may be outside of the marine vessel 150, whereby the mariner 152B is able to remote control the marine vessel 150 (which may then an unmanned or autonomous ship) or at least additionally monitor (as a pilot guiding ships through hazardous waters but residing in a piloting station or another remote-control station, for example, and maybe also control) the marine vessel 150 in addition to or instead of the mariner 152A aboard. The pilot 152B may thus better give guidance to the mariner 152A on board” (Paragraph 0073). As stated in Kim, “an aspect of the present disclosure is to provide a method for controlling operations of a UAV to stably operate the UAV by operating the UAV within a limited range and an electronic device for supporting the same” (Paragraphs 0003-0007).
Vanhakartano does not specifically state set a movable range of the
However, Bradski teaches set a movable range of the
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with setting a movable range of the vessel object of Vanhakartano and Bradski with a reasonable expectation of success. One of ordinary skill in the art would understand that Vanhakartano and Bradski both discuss using augmented reality for vehicles. One would have been motivated to combine as this allows the vehicle to steer towards a target while preventing collisions with obstacles (Vanhakartano - Paragraph 0073), (Bradski – Paragraph 0743).
Regarding claim 13, Vanhakartano discloses an augmented reality (AR) vessel maneuvering method, comprising (Paragraphs 0001, 0004).
All other limitations have been examined with respect to the system in claim 1. The method taught/disclosed in claim 13 can be clearly performed with the system of claim 1. Therefore, claim 13 is rejected under the same rationale.
Regarding claim 14, Vanhakartano discloses a non-transitory computer-readable storage medium having stored thereon machine-readable instructions that, when executed by one or more processors of an apparatus, cause the apparatus to perform a method comprising (Paragraphs 0001, 0004, 0021-0024).
All other limitations have been examined with respect to the system in claim 1. The non-transitory computer-readable storage medium taught/disclosed in claim 14 can be clearly performed with the system of claim 1. Therefore, claim 14 is rejected under the same rationale.
Claims 2-4, 7, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Vanhakartano, Kim, and Bradski, as applied to claim 1 above, and further in view of Grabowski (US 20180292213 A1, cited in a previous office action).
Regarding claim 2, Vanhakartano discloses the processing circuitry is further configured to (Paragraphs 0021-0025, 0035-0036);
the vessel object (Paragraphs 0025-0033, 0045, 0049-0051, Fig. 8).
Vanhakartano does not specifically state output a command to a navigation device used for navigating the vessel.
However, Grabowski teaches output a command to a navigation device used for navigating the vessel (Paragraphs 0022-0028; Navigation device is mapped to navigation systems (34)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with outputting a command to a navigation device to navigate the vessel of Grabowski with a reasonable expectation of success. One of ordinary skill in the art would understand that a vessel may need to change its trajectory based on collision risks. The pilot can use the user interface to output commands to the navigation device in order navigate the vessel and avoid collisions. One would have been motivated to combine Vanhakartano with Grabowski as this achieves collision avoidance. As stated in Grabowski, “accessing data and using local, transit-specific and shiphandling knowledge to gain the information needed to perform the three tasks of piloting: track-keeping; maneuvering and collision avoidance; and adherence to procedures and good practice” (Paragraph 0030).
Vanhakartano does not specifically state execute the navigation operation corresponding to the operation on the object.
However, Kim teaches execute the navigation operation corresponding to the operation on the object (Paragraph 0135).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with executing the navigation operation corresponding to the operation on the object of Kim with a reasonable expectation of success. One of ordinary skill in the art would understand that vehicles such as the vessel in Vanhakartano and the UAV in Kim need to stably maneuver without colliding into obstacles. Allowing an electronic device to control the vehicle by dragging a virtual object to a position and attitude is simple and allows the vehicle to be safely controlled within specified ranges. One would have been motivated to combine Vanhakartano with Kim as this achieves stable operation of the vehicle. As stated in Vanhakartano, “the mariner 152B may be outside of the marine vessel 150, whereby the mariner 152B is able to remote control the marine vessel 150 (which may then an unmanned or autonomous ship) or at least additionally monitor (as a pilot guiding ships through hazardous waters but residing in a piloting station or another remote-control station, for example, and maybe also control) the marine vessel 150 in addition to or instead of the mariner 152A aboard. The pilot 152B may thus better give guidance to the mariner 152A on board” (Paragraph 0073). As stated in Kim, “an aspect of the present disclosure is to provide a method for controlling operations of a UAV to stably operate the UAV by operating the UAV within a limited range and an electronic device for supporting the same” (Paragraphs 0003-0007).
Regarding claim 3, Vanhakartano discloses the processing circuitry is further configured to (Paragraphs 0021-0025, 0035-0036);
the vessel object (Paragraphs 0025-0033, 0045, 0049-0051, Fig. 8).
Vanhakartano does not specifically state determine a target value for the navigation device based on at least one of.
However, Grabowski teaches determine a target value for the navigation device based on at least one of (Paragraphs 0006-0007, 0022-0026; Target value is mapped to input commands).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with determining a target value for the navigation device of Grabowski with a reasonable expectation of success. One of ordinary skill in the art would understand that the vessel pilot needs to input target value control commands in order to navigate the ship and avoid collisions. One would have been motivated to combine Vanhakartano with Grabowski as this achieves safe and successful navigation of a vessel. As stated in Grabowski, “perform the three tasks of piloting: track-keeping; maneuvering and collision avoidance; and adherence to procedures and good practice developed over years of shiphandling, referred to as the ‘practice of good seamanship’” (Paragraph 0030).
Vanhakartano does not specifically state a position and an attitude of the object after the operation.
However, Kim teaches a position and an attitude of the object after the operation (Paragraph 0135).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with navigating a vehicle based on the position and attitude of the object after the operation of Kim with a reasonable expectation of success. One of ordinary skill in the art would understand that vehicles such as the vessel in Vanhakartano and the UAV in Kim need to stably maneuver and avoid collisions. Allowing an electronic device to control the vehicle by dragging a virtual object to a position and attitude is simple and allows the vehicle to be safely controlled within specified ranges. One would have been motivated to combine Vanhakartano with Kim as this achieves stable operation of the vehicle. As stated in Vanhakartano, “the mariner 152B may be outside of the marine vessel 150, whereby the mariner 152B is able to remote control the marine vessel 150 (which may then an unmanned or autonomous ship) or at least additionally monitor (as a pilot guiding ships through hazardous waters but residing in a piloting station or another remote-control station, for example, and maybe also control) the marine vessel 150 in addition to or instead of the mariner 152A aboard. The pilot 152B may thus better give guidance to the mariner 152A on board” (Paragraph 0073). As stated in Kim, “an aspect of the present disclosure is to provide a method for controlling operations of a UAV to stably operate the UAV by operating the UAV within a limited range and an electronic device for supporting the same” (Paragraphs 0003-0007).
Regarding claim 4, Vanhakartano discloses the AR vessel maneuvering system.
Vanhakartano does not specifically state the navigation device is a marine navigation device.
However, Grabowski teaches the navigation device is a marine navigation device (Paragraphs 0022-0028).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with a marine navigation device of Grabowski with a reasonable expectation of success. One of ordinary skill in the art would understand that a vessel may need to change its trajectory based on collision risks. The pilot can use the user interface to output commands to the marine navigation device in order navigate the vessel and avoid collisions. One would have been motivated to combine Vanhakartano with Grabowski as this achieves successful marine vessel navigation and collision avoidance. As stated in Grabowski, “accessing data and using local, transit-specific and shiphandling knowledge to gain the information needed to perform the three tasks of piloting: track-keeping; maneuvering and collision avoidance; and adherence to procedures and good practice” (Paragraph 0030).
Regarding claim 7, Vanhakartano discloses a vessel traveling on a target route and waypoints.
Vanhakartano does not specifically state the navigation device is a plotter implemented on the vessel; the target value is one of a target route and a waypoint for the plotter.
However, Grabowski teaches the navigation device is a plotter implemented on the vessel (Paragraphs 0023, 0047; Plotter is mapped to automatic radar plotting aid (ARPA));
the target value is one of a target route and a waypoint for the plotter (Paragraphs 0037-0047; The plotter has a target value because the WIAR device obtains the route and waypoint information from the ship’s navigational systems (34)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with a plotter, and a target route and waypoint for the plotter of Grabowski with a reasonable expectation of success. One of ordinary skill in the art would understand that a plotter assists the pilot in navigating a ship. A target route and waypoint for the plotter allows the pilot to visualize the route he/she is taking for the ship. One would have been motivated to combine Vanhakartano with Grabowski as this improves navigation ability. As stated in Grabowski, “Illustrative information that can be displayed/outputted includes navigation data, compass readings, auxiliary ship functions, weather information, ship ECDIS (electronic chart display & information system) monitors, ship ARPA (automated radar plotting aid) monitors” (Paragraph 0023).
Regarding claim 12, Vanhakartano discloses the image is displayed on a head-mounted display (Paragraphs 0025-0033);
the processing circuitry is further configured to (Paragraphs 0021-0025, 0035-0036);
generate the image including the vessel object by rendering the vessel object arranged at a position corresponding to a virtual three-dimensional space (Paragraphs 0021, 0025-0034, 0042-0047).
Vanhakartano does not specifically state set a viewpoint position and a line-of-sight direction according to a position and an attitude of the head-mounted display.
However, Grabowski teaches set a viewpoint position and a line-of-sight direction according to a position and an attitude of the head-mounted display (Paragraph 0046, 0056, Figs. 5, 6, and 9).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with setting a viewpoint position and a line-of-sight direction according to a position and attitude of the head-mounted display of Grabowski with a reasonable expectation of success. One of ordinary skill in the art would understand that the pilot on board the vessel and the shoreside personnel can share augmented reality images with each other. This allows the pilot and shoreside personnel to be synchronized with necessary information to ensure safe navigation and docking. One would have been motivated to combine Vanhakartano with Grabowski as this alleviates navigational transit and bridge resource management challenges. As stated in Grabowski, “The synchronization and shared mental models of the navigational transit could alleviate traditional bridge resource management challenges with differing mental models of the transit held by different members of the bridge watch team” (Paragraphs 0055-0056).
Claims 9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Vanhakartano, Kim, and Bradski, as applied to claim 1 above, and further in view of Behrendt (US 20210389765 A1, cited in a previous office action).
Regarding claim 9, Vanhakartano discloses the processing circuitry is further configured to (Paragraphs 0021-0025, 0035-0036);
acquire information associated with navigation of the vessel (Paragraphs 0016-0025);
determine a predicted position of the vessel after a predetermined time has elapsed based on the information associated with the navigation of the vessel (Paragraphs 0076-0077).
Vanhakartano does not specifically state generate the image including the vessel object representing the vessel at a position corresponding to the predicted position.
However, Behrendt teaches generate the image including the vessel object representing the vessel at a position corresponding to the predicted position (Claim 10).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Vanhakartano with generating an image including a vessel object at the predicted position of Behrendt with a reasonable expectation of success. One of ordinary skill in the art would understand that an image with predicted future positions of the vessel allows the user to more easily determine a heading or course of the vessel. One would have been motivated to combine Vanhakartano with Behrendt as this improves user understanding of navigating the vessel and updating navigation instructions. As stated in Behrendt, “the system advantageously displays a graphically representative model of the vessel, including proportionally accurate dimensions, to facilitate determining a heading or course of the vessel through the marine environment, and advantageously enables touchpoint input for more quickly and easily entering, revising, and updating navigation instructions including future headings at intermediate future waypoints” (Paragraph 0015).
Regarding claim 11, Vanhakartano discloses the information associated with the navigation of the vessel is at least one of: a target route and a waypoint of the vessel (Paragraph 0076-0077, Fig. 8; Waypoint is mapped to cross-lines 802).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to Matthew Ho whose telephone number is (571) 272-1388. The examiner can
normally be reached on Mon-Thurs 9:00-5:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Navid Z Mehdizadeh can be reached on (571)-272-7691. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications are available through Private PAIR only. For more information about the PAIR system, see https://ppairmy.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at (866) 217-9197 (tollfree). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call (800) 786-9199 (IN USA OR CANADA) or (571) 272-1000.
/MATTHEW HO/ Examiner, Art Unit 3669
/NAVID Z. MEHDIZADEH/Supervisory Patent Examiner, Art Unit 3669
Prosecution Insights