DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Amendment
Applicant’s preliminary amendment was received by the Office on 01 July 2025. Claims 1-35 have been presented in the application, of which, claims 1-15 are cancelled and claims 16-35 are new. Accordingly, pending claims 16-35 are addressed herein.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 20 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding claim 20, the phrase "in particular" renders the claim indefinite because it is unclear whether the limitation(s) following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 16-35 is/are rejected under 35 U.S.C. 103 as being unpatentable over Floyd-Jones et al. (US 2024/0009845 A1, hereinafter referred to as “Floyd-Jones”) and Thomasson et al. (US 2017/0372139 A1, hereinafter referred to as “Thomasson”).
Regarding claim 16, Floyd-Jones teaches a method for validating a predefined path of a robot, comprising: obtaining (via Fig. 1, element 122a, 122b, 124) a computer-implemented (via Fig. 1, element 128), three-dimensional environment model (Figs. 1-2, element 120; paragraphs 0055, 0070-0077, 0114, 0122); determining (Figs. 5-8, steps 504, 604, 704, 708, 804 via Fig. 2, element 264) a distance (clearance/margin; numeric value, e.g., millimeters, centimeters, inches) between a computer-implemented model (Figs. 1-2, element 112) of the robot (Figs. 1-2, element 102) and the environment model for different portions of the path (paragraphs 0010, 0060, 0065-0075, 0078, 0101, 0106-0111, 0137, 0146, 0155, 0165, 0187); visually displaying (Figs. 5-7 and 13-16, step 506, 606, 706, 1302, 1402, 1502, 1602) a virtual representation of the path (Figs. 21-28, elements 2108, 2208, 2308, 2408, 2508, 2608, 2708, 2808) […] for checking the path using a visualization device (Figs. 1-2, element 128, 128a; paragraphs 0010-0013, 0079, 0138-0140, 0147-0148, 0156, 0185-0192, 0207-0229); issuing (Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) a warning in the visual display for a portion of the path in response to a determination that the distance determined for the portion of the path falls within a predefined warning range (paragraphs 0010-0013, 0139, 0148, 0158, 0187-0192; visual indication using color or heat map corresponding to insufficient clearance or deviation from specified nominal clearance); issuing (Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) an all-clear message in the visual display for a portion of the path in response to a determination that the distance determined for the portion falls within a predetermined all-clear range (paragraphs 0010-0013, 0139, 0148, 0158, 0189-0192; visual indication using color or heat map corresponding to larger or sufficient clearance relative to nominal). Floyd-Jones is silent regarding visually displaying the virtual representation of the path in an augmented reality.
Thomasson teaches the known technique of visually displaying a virtual representation of a robot path in an augmented reality using a visualization device (head mounted transparent AIR display 122 or overlay on real-time images) so that the representation appears directly in the physical workspace (paragraphs 0008, 0022, 0028-0030, 0035-0043). It would have been obvious to a person having ordinary skill in the art prior to Applicant’s effective filing date to apply the known AR visualization technique taught by Thomasson to the clearance-determination and visual-indication system and method taught by Floyd-Jones. The Floyd-Jones system and method was ready for improvement as it presents clearance information on a conventional display, still requiring the user to mentally relate that information to the real workspace. Applying Thomasson’s AR technique yields the predictable result that the same indications are now overlaid in the operating environment, thereby improving accuracy and usability of path validation without unexpected results.
Regarding claim 17, Floyd-Jones teaches the method of claim 16, further comprising: detecting data of a real environment (Fig. 1, element 104) of the robot using a detection device (Figs. 1-2, elements 122a, 122b, 124; paragraphs 0065-0068, 0074-0076); wherein obtaining the environment model (Fis. 1-2, element 120) comprises determining the environment model based the detected data (paragraph 0074).
Regarding claim 18, Floyd-Jones teaches the method of claim 17, wherein at least one of: the detection device is a mobile detection device; the detection device is a portable detection device; the environment model is determined based on the detected data using at least one approximation of features detected with the aid of the detection device; or the environment model is determined based on the detected data using at least one approximation of points detected with the aid of the detection device (paragraphs 0074-0075; perception system 124 generates environmental model 120 as point cloud, occupancy grid, boxes or stream of voxels from raw sensor data from sensors 122a, 122b).
Regarding claim 19, Floyd-Jones teaches the method of claim 17, wherein at least one of: the detection device is arranged on the visualization device; the detection device is moved at least one of translationally or rotationally relative to the real environment in order to detect the data; or the detection device comprises at least one of: at least one non-contact measuring distance meter, at least one camera, or at least one image evaluation device (paragraphs 0074, 0105).
Regarding claim 20, Floyd-Jones teaches the method of claim 16, wherein at least one of: the environment model is determined on the basis of predefined nominal data, in particular CAD data, of the environment (paragraphs 0074-0076; environmental model 120 incorporates static object data known a priori); the environment model comprises at least one three-dimensional geometry primitive in a predefined relation to a real environmental obstacle (paragraphs 0074, 0108); a model of the robot is determined on the basis of predefined nominal data, in particular the predetermined path of the robot and/or CAD data of the robot (paragraph 0072; robot geometric models, GEOMODELS; Fig. 1, element 112); a measurement of the robot is determined and/or comprises at least one three- dimensional geometry primitive in a predetermined relation to a link of the robot (paragraphs 0066, 0108; sphere trees, meshes of links); or the model of the robot comprises a computer-implemented model of a robot- guided tool (Fig. 1, element 105c) or workpiece as a moving link of the robot (paragraphs 0012, 0066, 0085).
Regarding claim 21, Floyd-Jones teaches the method of claim 16, wherein the environment model is determined on the basis of at least one of: the robot (Fig. 1, element 102a/102b); a model of the robot (Fig. 1, element 112); or a selection (input or provided) of an environment area by a checking
Regarding claim 22, Floyd-Jones teaches the method of claim 21, wherein determining the environment model on the basis of the robot comprises determining based on data of the robot detected using a detection device (Fig. 1, element 120, 122a, 122b, 124; paragraphs 0068, 0074-0077; environment model 120 generated by perception system 124 from raw sensor data of operational environment 104 to include other robots 122b treated as obstacles).
Regarding claim 23, Floyd-Jones teaches the method of claim 16, wherein determining the distance between the model of the robot and the environment model for at least one of the portions of the path comprises determining (Figs. 5-8, step 504, 604, 804) the distance based on a minimum distance between an imaginary envelope of at least one movable link of the robot with an imaginary envelope of the entire environment of the robot described by the environment model, or an imaginary envelope of a selected partial region of the environment of the robot (paragraphs 0074, 0098-0099, 0106-0109; 0137; note that under BRI the term “imaginary envelope” is given its broadest reasonable meaning to include any virtual geometric representation, bounding volume, surface or approximation that encloses or represents an outer extent. Floyd-Jones teaches: swept volume, distance-field surface, sphere, bounding box, sphere, sphere tree, mesh and the like).
Regarding claim 24, Floyd-Jones teaches the method of claim 23, wherein: the imaginary envelope of the at least one movable link of the robot comprises an imaginary envelope of a plurality of movable links of the robot (paragraphs 0010-0013, 0106-0109, 0137, 0146, 0155); or the imaginary envelope of the at least one movable link of the robot is an imaginary envelope of all movable links of the robot (paragraphs 0010-0013, 0106-0109).
Regarding claim 25, Floyd-Jones teaches the method of claim 16, wherein the virtual representation of the path comprises at least one of: a path (Fig. 4, element 406; Figs. 25-28, elements 2508, 2608, 2708, 2808, 2810, 2812) of a robot-fixed reference point as it moves along the path; or a representation (Fig. 25-28, elements 2508, 2608, 2708a, 2710a, 2808, 2810, 2812) of at least one movable link (Figs. 25-26, element 2516, 2616, 2716) of the robot as it moves along the path (paragraphs 0011-0012, 0041, 0066, 0134, 0218-0229).
Regarding claim 26, Floyd-Jones teaches the method of claim 25, wherein at least one of: the path of the robot-fixed reference point comprises a continuous path (Figs. 4 and 21-28, elements 406, 2108, 2208, 2308, 2408, 2508, 2608, 2708, 2808) of the robot-fixed reference point (paragraphs 0011, 0054, 0134, 0207-0229); the robot-fixed reference point is and end effector (Fig. 1, element 105b) of the robot (paragraphs 0012, 0062, 0066, 0085, 0133); the representation of at least one movable link comprises a representation of a plurality of the movable links of the robot (Figs. 4, and 25-28, elements 405, 2516, 2616, 2716, 2816; paragraphs 0012, 0133-0134, 0220-0229); or the representation of at least one movable link comprises a simulation of the robot travelling on the path (Figs. 1-2, element 126, 262; paragraphs 0007-0008, 0011, 0016, 0078, 0106).
Regarding claim 27, Floyd-Jones teaches the method of claim 16, wherein visually displaying the virtual representation of the path comprises: visually displaying (Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) a portion of the path in a first virtual representation (color or heat map e.g., red/orange corresponding to insufficient clearance or deviation from specified nominal clearance) in response to the distance determined for that portion of the path falling within the warning range (paragraphs 0010, 0189-0192); and visually displaying (Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) the portion of the path in a second virtual representation (color or heat map e.g., green, corresponding to larger or sufficient clearance relative to nominal), different from first virtual representation, in response to the distance determined for the portion of the path falling within the all-clear range (paragraphs 0010, 0189-0192).
Regarding claim 28, Floyd-Jones teaches the method of claim 27, further comprising at least one of: visually displaying (Figs. 15-16, step 1502, 1602) the portion of the path in a third virtual representation (multi-shade map or multiple colors, e.g., dark red / light red, corresponding to amount of deviation from specified nominal clearance) in response to the distance determined for that portion of the path falling within a specified part of the warning range (paragraphs 0010, 0189-0192); and visually displaying (Figs. 15-16, step 1502, 1602) the portion of the path in a fourth virtual representation (multi-shade map or multiple colors, e.g., light green / dark green, corresponding to amount of sufficient clearance relative to specified nominal clearance) in response to the distance determined for the portion of the path falling within a specified part of the all-clear range (paragraphs 0010, 0189-0192).
Regarding claim 29, Floyd-Jones teaches the method of claim 16, further comprising at least one of: indicating at least one path parameter (speed, cost, path smoothing) during the visual displaying of the virtual representation of the path (paragraphs 0010, 0014-0015, 0100, 0140, 0180, 0187, 0207, 0210); or indicating a value (numerical values of minimum clearances) of the determined distance for at least one portion of the path during the visual displaying of the virtual representation of the path (paragraphs 0010, 0187, 0207, 0210).
Regarding claim 30, Floyd-Jones teaches the method of claim 29, wherein at least one of: the at least one portion of the path is a point (Figs. 21-22, element 2110, 2210) on the path (paragraphs 0013, 0054, 0187-0188, 0210); the at least one portion of the path is a portion of the path travelled on or approached in a simulated manner during the visual displaying (paragraphs 0106-0107); the at least one path parameter is a velocity (paragraphs 0100, 0140, 0180, 0183); the value of the determined distance is at least one of a global minimum distance or a distance for the at least one selected portion of the path (paragraphs 0012-0013, 0172-0174, 0187-0188); or indicating the path parameter or the value of the determined distance comprises indicating at least one of numerically,
Regarding claim 31, Floyd-Jones teaches the method of claim 16, further comprising at least one of: confirming validation of the path, performed by a user, using the visualized virtual representation and the issued warning or all-clear message (paragraphs 0008, 0015, 0080); modifying the path (paragraphs 0015, 0080, 0140-0141, 0179-0180); or travelling on the validated path with the robot (paragraphs 0070, 0142, 0151, 0161, 0169).
Regarding claim 32, Floyd-Jones teaches the method of claim 31, wherein at least one of: confirming validation of the path comprises validating at least a portion of the virtual representation for which a warning is issued (paragraphs 0008, 0015, 0080, 0140-0141); or modifying the path comprises modifying the portion of the path for which a warning is issued (paragraphs 0015, 0080, 0140-0141, 0179-0180; user adjusts edges/nodes corresponding to motions that have insufficient clearance indicated by color or heatmap).
Regarding claim 33, the combination of Floyd-Jones and Thomasson teaches the method of claim 31, further comprising: visually displaying (Floyd-Jones, Figs. 5-7, 11 and 13-16, step 506, 510-512, 606, 706, 1108-110, 1302, 1402, 1502, 1602) a virtual representation of the modified path (Floyd-Jones, Figs. 21-28, elements 2108, 2208, 2308, 2408, 2508, 2608, 2708, 2808) using the visualization device in the augmented reality (via Thomasson, Fig. 1, element 122; paragraphs 0008, 0022, 0028-0030, 0035-0043) for checking the modified path (Floyd-Jones, paragraphs 0010-0013, 0079, 0138-0141, 0147-0148, 0156 0179-0180, 0185-0192, 0207-0229); issuing (Floyd-Jones, Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) a warning in the visual display for a portion of the modified path in response to a determination that the distance determined for the portion of the modified path is within a predefined warning range (Floyd-Jones, paragraphs 0010-0013, 0139, 0148, 0158, 0187-0192; visual indication using color or heat map corresponding to insufficient clearance or deviation from specified nominal clearance); and issuing (Floyd-Jones, Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) an all-clear message in the visual display for a portion of the modified path in response to a determination that the distance determined for the portion of the modified path is within a predefined all-clear range (Floyd-Jones, paragraphs 0010-0013, 0139, 0148, 0158, 0189-0192; visual indication using color or heat map corresponding to larger or sufficient clearance relative to nominal).
Regarding claim 34, Floyd-Jones teaches a system for validating a predefined path of a robot, the system comprising: means (Figs. 1-2, elements 120, 122a, 122b, 124) for providing a computer-implemented three-dimensional environment model (paragraphs 0074-0076); means (Figs. 5-8, steps 504, 604, 704, 708, 804 via Fig. 2, element 264) for determining a distance (clearance/margin; numeric value, e.g., millimeters, centimeters, inches) between a computer-implemented model (Figs. 1-2, element 112) of the robot (Figs. 1-2, element 102) and the environment model for different portions of the path (paragraphs 0010, 0060, 0065-0075, 0078, 0101, 0106-0111, 0137, 0146, 0155, 0165, 0187); and a visualization device (Figs. 1-2, element 128, 128a) configured for visually displaying (Figs. 5-7 and 13-16, step 506, 606, 706, 1302, 1402, 1502, 1602) a virtual representation of the path (Figs. 21-28, elements 2108, 2208, 2308, 2408, 2508, 2608, 2708, 2808) […] for checking the path (paragraphs 0010-0013, 0079, 0138-0140, 0147-0148, 0156, 0185-0192, 0207-0229); wherein a warning is issued (Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) in the visual display for a portion of the path in response to a determination that the distance determined for the portion of the path falls within a predefined warning range (paragraphs 0010-0013, 0139, 0148, 0158, 0187-0192; visual indication using color or heat map corresponding to insufficient clearance or deviation from specified nominal clearance); and wherein an all-clear message is issued (Figs. 5-7 and 14-16, step 508, 608, 710, 712, 1402, 1502, 1602) in the visual display for a portion of the path in response to a determination that the distance determined for the portion falls within a predetermined all-clear range (paragraphs 0010-0013, 0139, 0148, 0158, 0189-0192; visual indication using color or heat map corresponding to larger or sufficient clearance relative to nominal). Floyd-Jones is silent regarding visually displaying the virtual representation of the path in an augmented reality.
Thomasson teaches the known technique of visually displaying a virtual representation of a robot path in an augmented reality using a visualization device (head mounted transparent AIR display 122 or overlay on real-time images) so that the representation appears directly in the physical workspace (paragraphs 0008, 0022, 0028-0030, 0035-0043). It would have been obvious to a person having ordinary skill in the art prior to Applicant’s effective filing date to apply the known AR visualization technique taught by Thomasson to the clearance-determination and visual-indication system and method taught by Floyd-Jones. The Floyd-Jones system and method was ready for improvement as it presents clearance information on a conventional display, still requiring the user to mentally relate that information to the real workspace. Applying Thomasson’s AR technique yields the predictable result that the same indications are now overlaid in the operating environment, thereby improving accuracy and usability of path validation without unexpected results.
Regarding claim 35, Floyd-Jones teaches a computer program comprising program code stored on a non-transitory, computer-readable storage medium, the program code, when executed by one or more computers, causing the one or more computers to carry out the method of claim 16 (paragraphs 0088-0093, 0113, 0230).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DALE MOYER whose telephone number is (571)270-7821. The examiner can normally be reached Monday-Friday 8am-5pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi H Tran can be reached at 571-272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Dale Moyer/Primary Examiner, Art Unit 3656
Prosecution Insights