DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, filed 06/08/2026, with respect to the rejection of claims under 35 USC 102 via Millard have been fully considered and are partially persuasive. Therefore, the previous rejection has been withdrawn. However, upon further consideration, a new ground of rejection is made in view of new found art.
In light of the amendment to claims 16, 18, 22 and 23 the previous 112(b) rejection is withdrawn.
Applicant argued that Millard ‘532 does not disclose computer-implemented model of the robot and the representation of Millard’s robot in a virtual environment does not constitute a computer-implemented model. This argument has been found to be not persuasive. Applicant’s arguments and support from the specification does not provide any particular as to what a computer-implemented model of the robot specifically requires. Even the drawings (being objected herein) do not provide any graphical description of what a computer-implemented model of both the robot and the environment would look. Per ordinary skill in the art, the Office interprets any virtual representation of captured real object as a computer-implemented model. Hence, Millard ‘532 virtual robot and environment constitute computer-implemented model per the claim language.
Applicant’s argument regarding the definition of augmented reality visualization device to which Millard ‘532 lacks is persuasive. However, in view of new found secondary art, a new 103 rejection is provided herewith.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore the “computer-implemented three-dimensional model of the environment”, “computer-implemented model of the robot” and” visually displaying a virtual representation of the planned first path in an augmented reality using a visualization device “must be shown or the feature(s) canceled from the claims 15 and 33 and corresponding dependent claims.
The …”visually displaying a virtual representation of the at least one second path using the visualization device in the augmented reality, together with the virtual representation of the first path; selecting a path from a set comprising the first path and the at least one second path based on a user input; and at least one of: modifying at least one of the first path or the at least one second path based on a user input, or traversing at least one of the selected path or the modified path with the robot…” must be shown or the feature(s) canceled from claim 19 and corresponding dependent claims.
No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 18 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In regards to claim 18 which depends upon claim 16. It not known which “quality criterion” the claim is referring. The term lacks antecedent basis and renders the claim indefinite.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 15-18, 24-29, and 31-34 are rejected under 35 U.S.C. 102(a) (2) as being anticipated by Ooba et al. (US 20210154844).
In regards to claim 15 - Ooba ‘844 discloses a method for planning a path of a robot (20, Fig. 1, 2), comprising: detecting data of a real environment of the robot with a detection device (31); determining a computer-implemented three-dimensional model of the environment based on the detected data (P.0020 “…augmented reality display device 12 of the present example captures an image in which an actual robot 20 and a peripheral device 22 arranged at the periphery of the actual robot 20 are shown, and displays a virtual robot 23 overlaid on the actual robot 20 shown in the captured image. …”); planning a first path of the robot based on the environment model and a computer-implemented model of the robot such that a collision between the robot model and the environment model is avoided (P.0021, Fig. 1 & 2, see movement of robot arm represented by reference 23 on display 12, “… the interference of the actual robot 20 with the peripheral device 22 can be prevented….”); and visually displaying a virtual representation of the planned first path in an augmented reality using a visualization device (12, Fig. 1 & 2).
PNG
media_image1.png
356
700
media_image1.png
Greyscale
In regards to claim 16 – Ooba ‘844 discloses the method of claim 15 wherein at least one of: determining the computer-implemented three-dimensional model of the environment comprises determining by approximation of features detected by the detection device (approximation movement of work piece and robot in 3-D modeling, Fig. 1 &3);
In regards to claim 17 - Ooba ‘844 discloses the method of claim 15, wherein planning the first path comprises optimizing specified quality criterion (optimizing specified criterion is for interference prevention P.0021, P. 0025).
In regards to claim 18 - Ooba ‘844 discloses the method of claim 16, wherein the quality criterion depends on at least one of a distance of the robot from the environment (P. 0025 “… interference detection … determine the intersection of the two convex polyhedrons formed by the projection…”, the intersection determination involves distance consideration from the environment).
In regards to claim 24 – Ooba ‘844 discloses wherein the detection device is at least one of at least one camera (31, Fig. 2).
In regards to claim 25 – Ooba ‘844 discloses the at least one camera comprises a 3D camera system (three-dimensional image sensor 11, P.0025)
In regards to claim 26 – Ooba ‘844 discloses method of claim 15, wherein at least one of: the environment model comprises at least one three-dimensional geometry primitive in a specified relation to a real environment obstacle (peripheral device 22 or W2)
In regards to claim 27 – The claim is drafted in alternative language. Since one of the alternative limitations is anticipated by Ooba as indicated above, claim 27 is also anticipated. Nevertheless, Ooba ‘844 also discloses CAD representation of the robot 20 (Figs. 1&2).
In regards to claim 28 - Ooba ‘844 discloses method of claim 15 wherein the environment model is determined based on at least one of; the robot (20, Fig. 1&2).
In regards to claim 29 - Ooba ‘844 discloses the method of claim 15, wherein at least one of: the visualization device is designed and configured to control the robot(P.0028).
In regards to claim 31 - Ooba ‘844 discloses the method of claim 15, further comprising outputting at least one parameter of at least one of the first path while visually displaying the virtual representation of at least one of the first path (Figs. 1&2, see augmented reality display of robot path and position).
In regards to claim 32 - Ooba ‘844 discloses the method of claim 31, wherein at least one of: the at least one parameter comprises at least one of a direction of travel of at least one of the first path (Figs. 1&2, see augmented reality display of robot path and travel direction).
In regards to claim 33 – Ooba ‘844 discloses a system for planning a path of a robot (20, Fig. 1, 2),, the system comprising: a detection device (31) for detecting data of a real environment of the robot; means for determining a computer-implemented three-dimensional model of the environment based on the detected data (P.0020 “…augmented reality display device 12 of the present example captures an image in which an actual robot 20 and a peripheral device 22 arranged at the periphery of the actual robot 20 are shown, and displays a virtual robot 23 overlaid on the actual robot 20 shown in the captured image. …”); means for planning a first path of the robot based on the environment model and a computer-implemented model of the robot such that a collision between the robot model and the environment model is avoided (P.0021, Fig. 1 & 2, see movement of robot arm represented by reference 23 on display 12, “… the interference of the actual robot 20 with the peripheral device 22 can be prevented….”); and a visualization device configured for visually displaying a virtual representation of the planned first path in an augmented reality (12, Fig. 1 & 2).
In regards to claim 34 – Ooba ‘844 discloses a computer program product for planning a path of a robot, the computer program product comprising program code stored on a non-transient, computer-readable storage medium, the program code, when executed on a computer, causing the computer to perform the method of claim 15 (Fig.3).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 15 – 34 are rejected under 35 U.S.C. 103 as being unpatentable over Millard et al. (US2022/0365532) in view of Deyle et al. (US 11,327,503).
Regarding claim 15, Millard teaches: A method for planning a path of a robot, comprising: detecting data of a real environment of the robot with a detection device (P.0019, Fig. 1A/B & 3); determining a computer-implemented three-dimensional model of the environment based on the detected data (Figs 1A/B & 3, P.0046); planning a first path of the robot based on the environment model and a computer-implemented model of the robot such that a collision between the robot model and the environment model is avoided (P. 0055); and visually displaying a virtual representation of the planned first path using a visualization device (element 640).
Millard ‘532 does not explicitly disclose that the virtual representation is represented via augmented reality (AR) format.
Deyle ‘503, in the same field of endeavor, discloses path planning and simulation for moving robot for avoiding obstacles. Deyle teaches that virtual representation of robot and the environment could be displayed via augmented reality platform, P.332 “…The virtual model can be presented to the human operator 2910 using augmented reality (AR) …”.
It would have been obvious for one of ordinary skill in the art at, before the filing date of the claimed invention, to apply Millard’ 532 virtual representation using augmented reality as taught by Deyle ‘503. The modification yields predictable result of providing additional means for displaying virtual reality representation in more realistic augmented reality format for users.
Regarding claim 16, Millard ‘532 discloses: wherein at least one of: the detection device is at least one of a mobile detection device or a portable detection device (element 206); determining the computer-implemented three-dimensional model comprises determining by approximation of features detected by the detection device (P.0045, Fig. 3).
Regarding claim 17, Millard ‘532 discloses wherein planning the first path comprises optimizing specified quality criterion (P.0012).
Regarding claim 18, Millard ‘532 discloses: wherein the quality criterion depends on at least one of: a time required for traversing the first path with the robot (Paragraph [55], "The route may be optimized, for example, to minimize the time"); an energy required for traversing the first path with the robot; or a distance of the robot from the environment (P.0055, “…The route may be optimized, for example, to minimize the time or distance for the robot to reach the goal location…”).
Regarding claim 19, Millard ‘532 discloses: planning at least one second path of the robot based on the environment model and the model of the robot such that a collision between the robot model and the environment model is avoided (element 144; P.0038); visually displaying a virtual representation of the at least one second path using the visualization device, together with the virtual representation of the first path (Figure 1B; P.0075, "To provide for interaction with a user, the features can be implemented on a computer having a display device … for displaying information to the user and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer. Additionally, such activities can be implemented via touchscreen flat-panel displays and other appropriate mechanisms."); selecting a path from a set comprising the first path and the at least one second path (P.0039, "adopts one of the candidate paths on which to initiate travel that optimizes the path cost function, e.g., by selecting a candidate path minimizes a cost that results from evaluating the path cost function") based on a user input; and at least one of: modifying at least one of the first path or the at least one second path based on a user input, or traversing at least one of the selected path or the modified path with the robot (P.0050, "The system 20 may learn the planning policy for a specific person (or any particular instance of another class of objects) based on prior experience interacting with and traveling with that person or object, or the planning policy may be based on parameters manually specified by the person via user input"; P.0056, "The local planner 238 can constantly update the path as the robot travels, and SO it may or may not generate a complete path that terminates at the goal location in a single pass").
Regarding claim 20, Millard ‘532 discloses: wherein modifying at least one of the first path or the at least one second path comprises modifying the selected path (P.0056, "The local planner 238 can constantly update the path as the robot travels, and so it may or may not generate a complete path that terminates at the goal location in a single pass").
Regarding claim 21, Millard ‘532 discloses: wherein planning at least one of the first path or the at least one second path comprises planning based on at least one of: at least one specified path point; or a specified initial path (element 118, 113; P.0035, "Accordingly, the robot 112 can use a local planner to refine the precise path or trajectory that it uses to maneuver in the environment 100, while still following the route 113 to reach goal location 120").
Regarding claim 22, Millard ‘532 discloses: wherein the at least one path point or the specified initial path is specified in advance by at least one of: approaching or traversing the at least one path point or the specified initial path with a real robot (element 116; P.0036); specifying with the aid of a simulation environment (P.0047); or specifying by offline programming (P.0043, "In other implementations, particular components of the system 200 may be separate from the robot 202, and may be located in either the same environment as the robot 202 or in a remote environment").
Regarding claim 23, Millard ‘532 discloses: wherein approaching or traversing the at least one path point or the specified initial path comprises hand-guided approaching or traversing (element 116; P.0036).
Regarding claim 24, Millard ‘532 discloses: wherein the detection device is at least one of: arranged on the visualization device; movable at least one of translationally, rotationally, or manually relative to the real environment for detecting the data; or comprises at least one of: at least one non-contact measuring distance meter (P.0045, "LIDAR system that uses light detection and ranging techniques to determine distances of objects from the robot in the environment"), at least one camera (element 206; P.0045).
Regarding claim 25, Millard ‘532 discloses: wherein at least one of: the at least one non-contact measuring distance meter comprises at least one of a lidar distance meter (P.0045, "LIDAR system that uses light detection and ranging techniques to determine distances of objects from the robot in the environment").
Regarding claim 26, Millard ‘532 discloses: wherein at least one of: the environment model comprises at least one three-dimensional geometry primitive in a specified relation to a real environment obstacle (P.0047).
Regarding claim 27, Millard ‘532 discloses: wherein the specified nominal data comprises CAD data of at least one of the robots or a measurement to the robot (P.0047).
Regarding claim 28, Millard ‘532 discloses: wherein the environment model is determined based on at least one of: the robot (P.0046, "The EAOD engine 214 is configured to generate a virtual environment representing all or a portion of the physical environment in which the robot 202 is currently located"); the model of the robot (P.0046, "The robot 202 can also be represented in the virtual environment at a virtual location corresponding to its current location in the real-world environment"); or a selection of an environment area by a planner (P.0046, "The EAOD engine 214 is configured to generate a virtual environment representing all or a portion of the physical environment in which the robot 202 is currently located").
Regarding claim 29, Millard ‘532 discloses: wherein at least one of: the visualization device comprises at least one of a handheld apparatus, a smartphone (P.0067, "The device 600 can also include mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other similar computing devices.").
Regarding claim 30, Millard ‘532 discloses: wherein the virtual representation of at least one of the first path comprises at least one of: a path of a robot-fixed reference point (Fig. 1B; Path A, element 120).
Regarding claim 31, Millard ‘532 discloses: further comprising outputting at least one parameter of at least one of the first path (Path A) or the at least one second path (Path B) while visually displaying the virtual representation of at least one of the first path or the at least one second path (Fig.1B; P.0075, "To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer. Additionally, such activities can be implemented via touchscreen flat-panel displays and other appropriate mechanisms.").
Regarding claim 32, Millard ‘532 discloses: wherein at least one of: the at least one parameter comprises at least one of a speed or a direction of travel of at least one of the first path (Fig. 1B; arrows point to goal).
Regarding claim 33, Millard ‘532 discloses: A system for planning a path of a robot, the system comprising: a detection device for detecting data of a real environment of the robot (element 206); means for determining a computer-implemented three-dimensional model of the environment based on the detected data (Figure 3; element 300; P.0046); means for planning a first path of the robot based on the environment model and a computer-implemented model of the robot such that a collision between the robot model and the environment model is avoided (P.0055); and visually displaying a virtual representation of the planned first path using a visualization device (element 640).
Millard ‘532 does not explicitly disclose that the virtual representation is represented via augmented reality (AR) format.
Deyle ‘503, in the same field of endeavor, discloses path planning and simulation for moving robot for avoiding obstacles. Deyle teaches that virtual representation of robot and the environment could be displayed via augmented reality platform, P.332 “…The virtual model can be presented to the human operator 2910 using augmented reality (AR) …”.
It would have been obvious for one of ordinary skill in the art at, before the filing date of the claimed invention, to apply Millard’ 532 virtual representation using augmented reality as taught by Deyle ‘503. The modification yields predictable result of providing additional means for displaying virtual reality representation in more realistic augmented reality format for users.
Regarding claim 34, Millard ‘532 discloses: a computer program product for planning a path of a robot, the computer program product comprising program code stored on a non-transient, computer- readable storage medium, the program code, when executed on a computer, causing the computer to perform the method of claim 15 (P.0073-0074).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Dickson et al. 2023/0194292 is directed to augmented reality for path planning and simulation for moving robot.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Supervisory Patent Examiner Khoi Tran whose telephone number is (571)272-6919. The examiner can normally be reached Mon-Thurs 7:00 AM to 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights