DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This final office action is a response to the reply received on 03/31/2026. Claims 1-3, 5 and 7 are pending. Claims 4 and 6 has been canceled. Claims 1-3 and 7 have been amended.
Information Disclosure Statement
The Information Disclosure Statements (IDS) received on 03/17/2026 and 06/03/2026 have been annotated and considered.
Response to Arguments
Applicant’s arguments with respect to the claims have been considered but are not persuasive. The applicant asserts on Page 7 of the remarks, that the reference Hiroshi does not disclose a text program, and that Padawer does not disclose the captured image information being displayed as information on the execution results for the work, or the displaying of the parameter information.
However, the rejection does not rely on Padawer for the captured image information or the parameter information indicating a cause of failure being displayed on the same screen. Similarly, the rejection does not rely on Hiroshi for the text program. Instead, Hiroshi supplies the captured image information and Padawer supplies the text program that can be followed line-by-line. Furthermore, Kudukoli discloses a single-window graphical user interface and a display of parameter information that indicates failure(s), and Miller supplied the live editing during execution of the program. Thus, the rejections are based on the combination of Hiroshi, Padawer, Kudukoli, and Miller (and Li) and not any individual reference to teach all of the limitations.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5, and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroshi et. al. (JP 2019171490 A, IDS) in view of Padawer et. al. (US 5124989 A), Kudukoli et. al. (US 20050268173 A1), and Miller (US 20100063608 A1).
Regarding Claim 1, Hiroshi discloses:
An industrial machine system that uses a sensor, which is a part of the system, to acquire information and performs work on a workpiece, the industrial machine system comprising: (See at least Figure 1 and ¶0022 via "an industrial robot 4 connected to the controller 3, and a camera 5 that captures the motion of the industrial robot 4" and ¶0027 via "An article handled by the industrial robot 4 (hereinafter referred to as a workpiece W) is, for example, a material such as glass or a semiconductor wafer.")
a storage unit configured to store an execution history of a program for performing work on the workpiece, wherein the storage unit stores information from the sensor and information regarding a processing result of the work on the workpiece, (See at least ¶0026 via "The storage unit 11 stores a management value 31 and operation history data" and ¶0033 via "The industrial robot 4 includes an encoder and a sensor that detect the rotational position, torque, temperature, and the like of each motor, and the encoder and sensor signals are output to the controller 3" and ¶0037 via "In other words, the temporary storage unit 22 stores the command and the torque data of the industrial robot 4 corresponding to the operation performed based on the command in the storage unit l1 as the temporary storage data 32." *Wherein the processing result is the torque associated with the command. Additionally see ¶0053 via "When photographing of the camera 5 is started, in step ST 23, photographing data is temporarily stored in time series")
the sensor includes a visual sensor, and the information from the sensor includes captured image information, (See at least "camera 5" and the captured image information being stored via ¶0053 via "when the operation of the industrial robot 4 is started, the management device 1 controls the camera 5 to photograph the industrial robot 4 in parallel with the process of storing the torque data in the storage unit 11 in time series. When shooting by the camera 5 is started, shooting data is temporarily stored in time series in step ST23.")
in a case where a failure has occurred in the processing result, information indicating a cause of the failure in the processing result is added to the execution history, and (See at least ¶0056 via "The abnormal data 33 is extracted from the temporary stored data 32. For example, the torque data which went back by predetermined time to the past is extracted from the generating time of the torque value (abnormal value), which separated from the management range" **Wherein the abnormal data is included within the execution history)
the information indicating the cause of the failure is derived, after execution of the program, by analyzing a cause of a failure that has occurred during the execution or a cause of a failure that may occur in a future, and indicates which data value in the processing result has caused the failure; a display device having an execution history display screen that displays the execution history, (See at least ¶0050 "Further, when it is determined that there is an abnormality in the torque data, an abnormal data accumulation process is performed to store abnormal torque data as abnormal data in the storage unit 11" and also ¶0054 via "Determining whether or not there is an abnormality is a process of determining whether or not the maximum torque is within a range from the lower limit value T2 to the upper limit value T1..." and furthermore, see ¶0059 via "The management device 1 can display imaging data related to abnormal values together with torque data. When one of the abnormal data 33 is selected as shown in FIG. 11, the photographic data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, it is possible to confirm not only the torque data but also the shooting data and to find the cause of the cause.")
wherein, the execution history display screen displays the execution history shown in (See at least ¶0035 via the display unit 13, ¶0037 via "the temporary storage unit 22 stores the torque data of the industrial robot 4 in the storage unit 11 in association with the command. In other words, the temporary storage unit 22 stores the command and the torque data of the industrial robot 4 corresponding to the operation performed based on the command in the storage unit 11 as the temporary storage data 32", as well as ¶0056 via "For example, torque data including a torque value deviating from the management width is defined as abnormal data 33. The abnormal data 33 is extracted from the temporary stored data 32." and additionally ¶0059 via "In the management device l, it is possible to display the shooting data related to the abnormal value together with the torque data…photographing data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, not only torque data but also photographing data can be confirmed and used for investigation of the cause.")
the captured image information is displayed as information on the execution results for the work, (See at least ¶0059 via "The management device 1 can display imaging data related to abnormal values together with torque data…photographing data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, not only torque data but also photographing data can be confirmed and used for investigation of the cause." and ¶0053 via "when the operation of the industrial robot 4 is started, the management device 1 controls the camera 5 to photograph the industrial robot 4 in parallel with the process of storing the torque data in the storage unit 11 in time series)
a user can find out which execution history caused the failure by observing the captured image information (See at least ¶0059 via "In the management device l, it is possible to display the shooting data related to the abnormal value together with the torque data…photographing data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, not only torque data but also photographing data can be confirmed and used for investigation of the cause." **Wherein Hiroshi discloses the photographing data being used to help determine the cause of failure)
in a case where a failure(See at least ¶0059 via "in FIG. 11, the photographic data associated with the selected abnormal data is reproduced and displayed on the display unit 13." and also see ¶0060 via "ln addition, when abnormality occurs multiple times, the torque data of the entire time period in which the abnormality has occurred a plurality of rimes can be displayed in time series and can be used to investigate the cause. Similarly, it is possible to reproduce the photographing data of the entire time period in which the torque data is displayed in time series.")
However, although Hiroshi discloses displaying execution history and execution results associated with failures, Modified Hiroshi does not explicitly disclose displaying the execution history in a text program on a single screen and user being able to follow the program line-by-line.
Nevertheless, Padawer--who is directed towards a method of debugging a computer program--discloses: shown in a text program (See at least Col. 2 Lines 45-48 via "There are three basic steps to create a useable program: conceptualize the function of the program, write computer readable lines of code, and debug the program")
a user can find out which execution history caused the failure by (See at least Col. 3 Lines. 32-36 via "Other debug commands which a user may execute include: a step command in which the program is executed one line at a time with the lines being displayed to the user as they are executed" as well as Col. 3 Lines. 17-22 via "After the user enters a debug command, step 204, a record of the command is created and stored and, if the command is one which executes a line in the program, a record of the line number arrived at in the program is created and stored as a history point record." and additionally Col. 2 Lines. 24-28 via "The ability to reexecute the debug commands and stopping their execution at selected lines in the program aids the user in locating errors After locating the errors, the user modifies the program").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Hiroshi in view of Padawer in order to display the execution history line-by-line to enable the identification of failures within the program: "a user may first read the program line-by-line to try and locate the errors" (Padawer Col. 1 Lines. 21-23). One would be motivated to incorporate the teachings of Padawer with the photographing data of Hiroshi that is used to help identify a cause of an abnormality in order to make it easier for the user to identify failures by executing the program one line at a time and storing the line number reached after executing a debug command (See Padawer Col. 2 Lines. 10-30 and Col. 1 Lines. 30-42) in order to allow the user to stop execution at selected lines and modify as needed, thus, improving failure diagnosis in light of a user reading line-by-line being difficult if the program contains many loops, variables, etc: "Following the flow of a program by reading it line-by-line is difficult, even in a relatively simple program. If the program contains many loops, subroutines based on variables, or calls, the user may not be able to trace the sequence of execution, and hence the function of each line. (Padawer Col. 1 Lines. 23-27).
However, although Hiroshi discloses a display device and detecting when a failure occurs, Modified Hiroshi does not explicitly disclose the program editing device.
Nevertheless, Kudukoli--who is directed towards programmatically analyzing a program--discloses: on a single screen (See at least ¶0055 via "A GUI may comprise a single window having one or more GUI Elements")
and in a case where a failure in displaying by the display device has occurred in processing by the program, parameter information indicating a cause of the failure in the displaying is displayed on the same execution history display screen; (See at least ¶0139 via " Clicking on a particular failure occurrence shown under a test may cause the GUI panel to display text at the bottom of the panel to explain why the test failed for that occurrence…and the text at the bottom informs the user that this occurrence corresponds to a wire that has a segment with a length less than a specified minimum pixel length. As described below, in one embodiment the graphical program analyzer may also be operable to automatically display the portion of the graphical program that failed the test." *Wherein the example of the parameter is the wire length. Also see ¶0144 via "graphical program analyzer may also be operable to display portions of the graphical program that correspond to the displayed results of the analysis…display the graphical program portion that corresponds to each failure occurrence of the tests that were performed. This may allow the user to easily correlate the analysis results with the graphical program itself." and ¶0055 via "single window")
a program editing device having a program editing screen configured to allow editing of the program, (See at least Figure 5 and ¶0112 via "editor window for editing the block diagram of a LabVIEW graphical program that performs a measurement function")
wherein in a case where a failure in editing by the program editing device occurs in processing by the program, the execution history of the program is read on the program editing screen of the program editing device so as to reproduce a state of work on the workpiece, and parameter information indicating a cause of the failure in the editing is displayed on the program editing screen, and (See at least ¶0138 via "It is possible that each test may be performed and may fail multiple times during the analysis of a graphical program. In one embodiment, the user may be able to view each failure occurrence. (As described above, in another embodiment the user may specify a "Maximum Reported Failures" parameter to specify a maximum number of failure occurrences to report.) For example, in the GUI panel of FIG. 11, the user may click the "+" and "-" signs beside each test in the results list to expand or collapse the failure occurrences for that test." and also ¶0139 via "As described below, in one embodiment the graphical program analyzer may also be operable to automatically display the portion of the graphical program that failed the test." as well as ¶0144 via "…display the graphical program portion that corresponds to each failure occurrence of the tests that were performed" and ¶0145 via "In one embodiment the graphical program analyzer may also highlight or call the user's attention to the relevant portion of the block diagram, e.g., may highlight one or more nodes, wires, or other objects in the block diagram that caused the test to fail. This may enable the user to easily. identify and modify the problem area of the block diagram if desired.")
a user can edit the program for a robot to perform work prior to the execution of the program and can modify a part of the program to test a partial program related to the modified part (See at least ¶0145 via "In one embodiment the graphical program analyzer may also highlight or call the user's attention to the relevant portion of the block diagram, e.g., may highlight one or more nodes, wires, or other objects in the block diagram that caused the test to fail. This may enable the user to easily. identify and modify the problem area of the block diagram if desired.")
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Hiroshi in view of Kudukoli's program analysis and editing in order to mitigate potential failures or errors that are based on an issue in the program itself by simplifying the user interface so the user is able to easily make adjustments: "Modern graphical programming development environments allow users to create large and complex graphical programs. It can sometimes be difficult for users to evaluate a graphical program simply by looking at its block diagram. Thus, it may be desirable to enable graphical programs to be programmatically analyzed in various ways, e.g., to provide the user with information about the graphical program or inform the user of potential problems in the graphical program." [Kudukoli ¶0009]. Furthermore, one of ordinary skill in the art would have been motivated to incorporate the text program of Padawer along with the photographic data of Hiroshi within a single window/screen having one or more GUI elements such as taught by Kudukoli in order to make it easier for a user to diagnose the cause of failure by being able to correlate the analysis results without having to switch between multiple screens: "…graphical program analyzer may also be operable to display portions of the graphical program that correspond to the displayed results of the analysis…display the graphical program portion that corresponds to each failure occurrence of the tests that were performed…allow the user to easily correlate the analysis results with the graphical program itself" [Kudukoli ¶0144].
However, Modified Hiroshi does not explicitly disclose the live editing of the program during execution.
Nevertheless, Miller--who is directed towards a program method for a machine tool--discloses: test a partial program related to the modified part even during the execution of the program (See at least ¶0057 via "Also, as previously discussed, the part program 111 can be edited while the part program is running. More specifically, if a part program is running and the operator edits a coordinate such as an x-axis value, the process will be updated in "real-time" such that the next time the motion instruction looks to the register holding the x-axis value, it will find and use the updated value").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify the industrial machine system as disclosed by Modified Hiroshi in view of the editing of the program while it is running such as in Miller, in order to allow the most updated data/program to be used while avoiding stopping a running program and needing to restart it: "the process will be updated in "real-time" such that the next time the motion instruction looks to the register holding the x-axis value, it will find and use the updated value" [Miller ¶0057].
Regarding Claim 3, Modified Hiroshi discloses the industrial machine system according to Claim 1.
Furthermore, Hiroshi discloses: wherein the sensor further comprises a force sensor, (See at least ¶0033 via "The industrial robot 4 includes an encoder and a sensor for detecting the rotational position, the torque, the temperature, and the like of each motor, and the signals of the encoder and the sensor are output to the controller 3.")
the industrial machine system performs work on the workpiece while detecting a load using the force sensor, information from the force sensor includes information regarding an output from the force sensor, and (See at least ¶0038 via "Alternatively, both the torque data and the position data may be stored as temporary stored data 32. That is, one or both of the torque data and the position data can be stored as the operation history data of the industrial robot 4.")
the information regarding the processing result includes information regarding a result of processing of the output from the force sensor (See at least ¶0037 via "In other words, the temporary storage unit 22 stores the command and the torque data of the industrial robot 4 corresponding to the operation performed based on the command in the storage unitl1 as the temporary storage data 32." *Wherein the processing result is the torque associated with the command).
Regarding Claim 5, Modified Hiroshi discloses the industrial machine system according to Claim 1.
Furthermore, Hiroshi discloses: further comprising: an operation panel having an input unit configured to allow the user to perform an input when the program is being executed, (See at least ¶0035 via "The input unit 14 is an input unit such as a keyboard, a mouse, or a touch panel provided on a display screen". Additionally see at least ¶0042 via "In the management screen G1, a list 50, a torque data specification field 51, and a management value determination button 52 are displayed. In the torque data designation column 51, an input for specifying an operation for torque monitoring can be input.")
wherein in a case where a failure has occurred in processing by the program executed with an input value inputted via the operation panel by the user, parameter information indicating a cause of the failure is notified to the user (See at least ¶0061 via "When the abnormal data processing unit 23 detects that the occurrence frequency of the abnormal data 33 exceeds a preset reference frequency, the abnormal data processing unit 1 performs a process of notifying the occurrence of the occurrence of the abnormal data. For example, a process for printing an alert from a printer connected to the network 6 or a notification to the registered mail address is reported").
Regarding Claim 7, Hiroshi discloses:
An industrial machine system that uses a sensor, which is a part of the system, to acquire information and performs work on a workpiece, wherein the sensor includes a visual sensor, and the information from the sensor includes captured image information, the industrial machine system comprising: (See at least Figure 1 and ¶0022 via "an industrial robot 4 connected to the controller 3, and a camera 5 that captures the motion of the industrial robot 4" and ¶0027 via "An article handled by the industrial robot 4 (hereinafter referred to as a workpiece W) is, for example, a material such as glass or a semiconductor wafer." -- also see "camera 5" and the captured image information being stored via ¶0053 via "when the operation of the industrial robot 4 is started, the management device 1 controls the camera 5 to photograph the industrial robot 4 in parallel with the process of storing the torque data in the storage unit 11 in time series. When shooting by the camera 5 is started, shooting data is temporarily stored in time series in step ST23.")
a storage unit configured to store an execution history of a program for executing work on the workpiece; (See at least ¶0026 via "The storage unit 11 stores a management value 31 and operation history data")
a display device having an execution history display screen that displays the execution history; and (See at least ¶0035 via "The display unit 13 is, for example, a liquid crystal display, and includes various management screens displayed by the management program…")
wherein the execution history display screen displays the execution history shown in (See at least ¶0035 via the display unit 13, ¶0037 via "the temporary storage unit 22 stores the torque data of the industrial robot 4 in the storage unit 11 in association with the command. In other words, the temporary storage unit 22 stores the command and the torque data of the industrial robot 4 corresponding to the operation performed based on the command in the storage unit 11 as the temporary storage data 32", as well as ¶0056 via "For example, torque data including a torque value deviating from the management width is defined as abnormal data 33. The abnormal data 33 is extracted from the temporary stored data 32." and additionally ¶0059 via "In the management device l, it is possible to display the shooting data related to the abnormal value together with the torque data…photographing data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, not only torque data but also photographing data can be confirmed and used for investigation of the cause.")
the captured image information is displayed as information on the execution results for the work, (See at least ¶0059 via "The management device 1 can display imaging data related to abnormal values together with torque data…photographing data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, not only torque data but also photographing data can be confirmed and used for investigation of the cause." and ¶0053 via "when the operation of the industrial robot 4 is started, the management device 1 controls the camera 5 to photograph the industrial robot 4 in parallel with the process of storing the torque data in the storage unit 11 in time series.)
in a case where a failure(See at least ¶0059 via "in FIG. 11, the photographic data associated with the selected abnormal data is reproduced and displayed on the display unit 13." and also see ¶0060 via "ln addition, when abnormality occurs multiple times, the torque data of the entire time period in which the abnormality has occurred a plurality of rimes can be displayed in time series and can be used to investigate the cause. Similarly, it is possible to reproduce the photographing data of the entire time period in which the torque data is displayed in time series.")
a user can find out which execution history caused a failure by observing the captured image information (See at least ¶0059 via "In the management device l, it is possible to display the shooting data related to the abnormal value together with the torque data…photographing data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, not only torque data but also photographing data can be confirmed and used for investigation of the cause." **Wherein Hiroshi discloses the photographing data being used to help determine the cause of failure)
a cause of a failure at a time of execution or a cause of a failure that may occur in a future is analyzed, information indicating the analyzed cause of the failure is displayed on the program editing screen, and (See at least ¶0050 "Further, when it is determined that there is an abnormality in the torque data, an abnormal data accumulation process is performed to store abnormal torque data as abnormal data in the storage unit 11" and also ¶0054 via "Determining whether or not there is an abnormality is a process of determining whether or not the maximum torque is within a range from the lower limit value T2 to the upper limit value T1 (l. e., whether or not the maximum torque is within the range of the management width), and determining whether or not the minimum torque is within a range from the lower limit value T4 to the upper limit value T3." and also see ¶0059 via "In the management device l, it is possible to display the shooting data related to the abnormal value together with the torque data. When one of the abnormal data 33 is selected as shown in FIG. 11, the photographic data associated with the selected abnormal data is reproduced and displayed on the display unit 13. Therefore, it is possible to confirm not only the torque data but also the shooting data and to find the cause of the cause.)
However, although Hiroshi discloses displaying execution history and execution results associated with failures, Modified Hiroshi does not explicitly disclose displaying the execution history in a text program on a single screen and user being able to follow the program line-by-line.
Nevertheless, Padawer--who is directed towards a method of debugging a computer program--discloses: shown in a text program (See at least Col. 2 Lines 45-48 via "There are three basic steps to create a useable program: conceptualize the function of the program, write computer readable lines of code, and debug the program")
a user can find out which execution history caused the failure by (See at least Col. 3 Lines. 32-36 via "Other debug commands which a user may execute include: a step command in which the program is executed one line at a time with the lines being displayed to the user as they are executed" as well as Col. 3 Lines. 17-22 via "After the user enters a debug command, step 204, a record of the command is created and stored and, if the command is one which executes a line in the program, a record of the line number arrived at in the program is created and stored as a history point record." and additionally Col. 2 Lines. 24-28 via "The ability to reexecute the debug commands and stopping their execution at selected lines in the program aids the user in locating errors After locating the errors, the user modifies the program").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Hiroshi in view of Padawer in order to display the execution history line-by-line to enable the identification of failures within the program: "a user may first read the program line-by-line to try and locate the errors" (Padawer Col. 1 Lines. 21-23). One would be motivated to incorporate the teachings of Padawer with the photographing data of Hiroshi that is used to help identify a cause of an abnormality in order to make it easier for the user to identify failures by executing the program one line at a time and storing the line number reached after executing a debug command (See Padawer Col. 2 Lines. 10-30 and Col. 1 Lines. 30-42) in order to allow the user to stop execution at selected lines and modify as needed, thus, improving failure diagnosis in light of a user reading line-by-line being difficult if the program contains many loops, variables, etc: "Following the flow of a program by reading it line-by-line is difficult, even in a relatively simple program. If the program contains many loops, subroutines based on variables, or calls, the user may not be able to trace the sequence of execution, and hence the function of each line. (Padawer Col. 1 Lines. 23-27).
However, although Hiroshi discloses a display device and detecting when a failure occurs, Modified Hiroshi does not explicitly disclose the program editing device.
Nevertheless, Kudukoli--who is directed towards programmatically analyzing a program--discloses: on a single screen (See at least ¶0055 via "A GUI may comprise a single window having one or more GUI Elements")
a program editing device having a program editing screen configured to allow editing of the program, (See at least Figure 5 and ¶0112 via "editor window for editing the block diagram of a LabVIEW graphical program that performs a measurement function")
in a case where a failure in displaying by the display device has occurred in processing by the program, parameter information indicating a cause of the failure in the displaying is displayed on the same execution history display screen, (See at least ¶0139 via " Clicking on a particular failure occurrence shown under a test may cause the GUI panel to display text at the bottom of the panel to explain why the test failed for that occurrence…and the text at the bottom informs the user that this occurrence corresponds to a wire that has a segment with a length less than a specified minimum pixel length. As described below, in one embodiment the graphical program analyzer may also be operable to automatically display the portion of the graphical program that failed the test." *Wherein the example of the parameter is the wire length. Also see ¶0144 via "graphical program analyzer may also be operable to display portions of the graphical program that correspond to the displayed results of the analysis…display the graphical program portion that corresponds to each failure occurrence of the tests that were performed. This may allow the user to easily correlate the analysis results with the graphical program itself." and ¶0055 via "single window")
in a case where a failure in editing by the program editing device has occurred in processing by the program, the execution history of the program is read on the program editing screen of the program editing device so as to reproduce a state of work on the workpiece, (See at least ¶0138 via "It is possible that each test may be performed and may fail multiple times during the analysis of a graphical program. In one embodiment, the user may be able to view each failure occurrence. (As described above, in another embodiment the user may specify a "Maximum Reported Failures" parameter to specify a maximum number of failure occurrences to report.) For example, in the GUI panel of FIG. 11, the user may click the "+" and "-" signs beside each test in the results list to expand or collapse the failure occurrences for that test." and also ¶0139 via "As described below, in one embodiment the graphical program analyzer may also be operable to automatically display the portion of the graphical program that failed the test." as well as ¶0144 via "…display the graphical program portion that corresponds to each failure occurrence of the tests that were performed" and ¶0145 via "In one embodiment the graphical program analyzer may also highlight or call the user's attention to the relevant portion of the block diagram, e.g., may highlight one or more nodes, wires, or other objects in the block diagram that caused the test to fail. This may enable the user to easily. identify and modify the problem area of the block diagram if desired.")
a user can edit the program for a robot to perform work prior to the execution of the program and can modify a part of the program to test a partial program related to the modified part (See at least ¶0145 via "In one embodiment the graphical program analyzer may also highlight or call the user's attention to the relevant portion of the block diagram, e.g., may highlight one or more nodes, wires, or other objects in the block diagram that caused the test to fail. This may enable the user to easily. identify and modify the problem area of the block diagram if desired.")
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Modified Hiroshi in view of Kudukoli's program analysis and editing in order to mitigate potential failures or errors that are based on an issue in the program itself by simplifying the user interface so the user is able to easily make adjustments: "Modern graphical programming development environments allow users to create large and complex graphical programs. It can sometimes be difficult for users to evaluate a graphical program simply by looking at its block diagram. Thus, it may be desirable to enable graphical programs to be programmatically analyzed in various ways, e.g., to provide the user with information about the graphical program or inform the user of potential problems in the graphical program." [Kudukoli ¶0009]. Furthermore, one of ordinary skill in the art would have been motivated to incorporate the text program of Padawer along with the photographic data of Hiroshi within a single window/screen having one or more GUI elements such as taught by Kudukoli in order to make it easier for a user to diagnose the cause of failure by being able to correlate the analysis results without having to switch between multiple screens: "…graphical program analyzer may also be operable to display portions of the graphical program that correspond to the displayed results of the analysis…display the graphical program portion that corresponds to each failure occurrence of the tests that were performed…allow the user to easily correlate the analysis results with the graphical program itself" [Kudukoli ¶0144].
However, Modified Hiroshi does not explicitly disclose the live editing of the program during execution.
Nevertheless, Miller--who is directed towards a program method for a machine tool--discloses: test a partial program related to the modified part even during the execution of the program (See at least ¶0057 via "Also, as previously discussed, the part program 111 can be edited while the part program is running. More specifically, if a part program is running and the operator edits a coordinate such as an x-axis value, the process will be updated in "real-time" such that the next time the motion instruction looks to the register holding the x-axis value, it will find and use the updated value").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify the industrial machine system as disclosed by Modified Hiroshi in view of the editing of the program while it is running such as in Miller, in order to allow the most updated data/program to be used while avoiding stopping a running program and needing to restart it: "the process will be updated in "real-time" such that the next time the motion instruction looks to the register holding the x-axis value, it will find and use the updated value" [Miller ¶0057].
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroshi et. al. (JP 2019171490 A, IDS), Kudukoli et. al. (US 20050268173 A1), and Miller (US 20100063608 A1) in view of Li et. al. (US 20200160616 A1).
Regarding Claim 2, Modified Hiroshi discloses the industrial machine system according to Claim 1.
Furthermore, Hiroshi discloses: wherein, the industrial machine system detects a workpiece using the visual sensor and performs work on the detected workpiece, and (See at least Figures 3 and 4. Also see ¶0034 which explains that when the hand of the robot (hand unit 41) is extended such as in Figure 3 (or top-down view Figure 4), it is in the range of the camera. The workpiece W is detected by the Camera 5 as it is being held in the figures 3-4 by extended hand 41. Additionally, see ¶0027 via "An article handled by the industrial robot 4 (hereinafter referred to as a workpiece W) is, for example, a material such as glass or a semiconductor wafer.")
PNG
media_image1.png
262
474
media_image1.png
Greyscale
However, modified Hiroshi does not explicitly disclose the non-detection of the object, and the position or posture information from the visual sensor.
Nevertheless, Li--who is in the same field of endeavor--discloses: the information regarding the processing result includes information regarding a detection result of the workpiece and information regarding non-detection/detection error of the workpiece (See at least [0061] Since an error may be present in the detected object and the estimated 3D pose, the 3D model alignment apparatus 100 additionally adjusts the object of the 2D input image and the target 3D model based on the estimated 3D pose, the feature point of the object in the 2D input image, and the feature point of the target 3D model. In an example, the adjustment may refer to calibration or correction").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify the industrial machine system disclosed by Modified Hiroshi in view of the information regarding a detection error of the workpiece disclosed by Li in order to make adjustments to allow the robot to continue operating with accurate and updated position information of both the robot and target object: "Accordingly, the 3D model alignment apparatus 100 may further enhance the accuracy of the estimated 3D pose, the target 3D model, and the object included in the 2D input image." [¶0061 Li].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAYLA RENEE DOROS whose telephone number is (703)756-1415. The examiner can normally be reached Generally: M-F (8-5) EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.D./Examiner, Art Unit 3657
/ABBY LIN/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights