DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a Final Office Action on the merits. Claims 1, 4-6, 8-12, 14-16, 18, and 21-28 are currently pending and are addressed below.
Response to Amendment
1. The amendment filed 02/03/2026 has been entered. Claims 1, 4-6, 8-12, 14-16, 18, and 21-28 remain pending in the application. Applicant’s amendments to the claims have overcome each 112(b) and objection previously set forth in the Non-Final Office Action mailed November 07, 2025.
Response to Arguments
2. Regarding the rejection made under 35 USC 102, the Applicant’s amendments and arguments have been fully considered but are moot because of the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Additionally, regarding method claims 23, 25, and 26, the examiner would like to point out that the phrase “sending, by the information processing portion, first information to an operating device if the operating device is connected…” is a conditional limitation. This means that the functionality or operation of the invention is dependent on certain conditions or circumstances. In this case, sending, by the information processing portion, first information is done if the operating device is connected. Broadly interpreted, if the condition for performing a contingent step is not satisfied, the performance recited by the step need not be carried out in order for the claimed method to be perform. The examiner recommends updating the phrase to make it a required limitation.
Claim Interpretation
3. The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
4. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
5. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“information processing portion” in claims 1, 4, 5, 6, 8, 9, 12, 14, 15, 16, 18, 21, 23, 24, 25, 26, and 27.
“storage portion” in claim 6.
“communication portion” in claim 11.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
A review of the specification shows the following appears to be the corresponding structure described in the specification for the 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph limitations:
“information processing portion” in claims 1, 4, 5, 6, 8, 9, 12, 14, 15, 16, 18, 21, 23, 24, 25, 26, and 27 corresponds to “the CPU 201 functions…as an information processing portion 252” [0053] and Fig. 4A.
“storage portion” in claim 6 corresponds to “memory (ROM) 202 or a random access memory (RAM) 203 or a hard disk drive (HDD) 204” [0047] and Fig. 3.
“communication portion” in claim 11 corresponds to “wireless communication unit 208 includes an antenna and a communication circuit” [0115] and Fig. 14.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1, 4-5, 8-12, 14-15, 18, 21-26, and 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 20200376680, hereinafter Wang) in view of Sugaya (US 20190221037, hereinafter Sugaya).
Regarding claim 1, Wang teaches an information processing apparatus (see at least Fig. 2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”) comprising:
an information processing portion (see at least Figs. 1-2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”; [0044]: “The mobile robot 20 shown in FIG. 2 includes the truck 50, the robot arm 60, a communication section 22, and a second detecting section 70.”; [0078]: “Examples of the processor shown in FIG. 5 include a CPU (Central Processing Unit).”),
wherein the information processing portion is configured to send first information to an operating device if the operating device is connected to the information processing apparatus (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on a motion of a first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”), and
wherein the information processing portion is configured to cause a display portion included in the operating device to display, based on the first information, the motion of the first robot using a model corresponding to the first robot, after the operating device is connected to the information processing apparatus and the first information is sent from the information processing apparatus to the operating device (see at least Figs. 6-9 and [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”).
Wang fails to explicitly teach displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to display, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 4, modified Wang teaches the limitations of claim 1. Wang further teaches wherein the information processing portion is configured to cause the display portion to display a model corresponding to an object around the first robot (see at least Figs. 2-3, 6-14, and [0038]: “Examples of a detection method for the marker M1 by a marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the portable teaching device 40 to the marker M1 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0061]: “Examples of a detection method for the marker M2 by the marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the mobile robot 20 to the marker M2 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0070]: “Any other equipment or the like may be provided in the mobile robot 20. Specifically, examples of the equipment include an imaging section that images a work target object, the mobile robot 20, or the periphery of the mobile robot 20 and various sensors such as a force sensor that detects an external force applied to the robot arm 60.”).
Wang fails to explicitly teach displaying a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays a three-dimensional model (see at least Fig. 3 and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to display a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation for the user to spatiality visualize.
Regarding claim 5, modified Wang teaches the limitations of claim 1. Wang further teaches wherein the information processing portion is configured to cause the display portion to further display captured-image data obtained by an image capture apparatus capturing an image, the image capture apparatus being disposed on the first robot (see at least Figs. 2-3, 6-14, and [0038]: “Examples of a detection method for the marker M1 by a marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the portable teaching device 40 to the marker M1 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0059]: “Therefore, the second detecting section 70 includes the marker detecting section 72.”; [0061]: “Examples of a detection method for the marker M2 by the marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the mobile robot 20 to the marker M2 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0063]: “With such a second detecting section 70, it is possible to measure the present position of the mobile robot 20 even with a relatively simple configuration. Accordingly, it is possible to achieve a reduction in the cost and a reduction in the size of the second detecting section 70.”; [0070]: “Any other equipment or the like may be provided in the mobile robot 20. Specifically, examples of the equipment include an imaging section that images a work target object, the mobile robot 20, or the periphery of the mobile robot 20 and various sensors such as a force sensor that detects an external force applied to the robot arm 60.”).
Regarding claim 8, modified Wang teaches the limitations of claim 1. Wang further teaches wherein the information processing portion is configured to send the first information and second information to the operating device if the operating device is connected to the information processing apparatus (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on the motion of the first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”), the second information being information on a motion of a second robot that may be operated by the operating device (see at least [0025]: “The robot system 1 may include one mobile robot 20. However, in FIG. 1, the robot system 1 includes a plurality of mobile robots 20.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”; Fig. 13 and [0137]: “As shown in FIG. 13, a planned moving track in future of the mobile robot 201 is displayed on the display section 43 as a line L3. By displaying such a line L3, the user U can intuitively grasp a moving destination of the mobile robot 201. Consequently, since teaching work adapted to the moving destination can be performed, it is possible to achieve efficiency of the teaching work.”; [0138]: “About the mobile robots 202 to 206 other than the mobile robot 201 selected by the selection operation, moving tracks may be displayed on the display section 43 as lines.”).
Regarding claim 9, modified Wang teaches the limitations of claim 8. Wang further teaches wherein the information processing portion is configured to cause the display portion to display, based on the second information, the motion of the second robot corresponding to the second robot (see at least Figs. 2-3, 6-14, and [0025]: “The robot system 1 may include one mobile robot 20. However, in FIG. 1, the robot system 1 includes a plurality of mobile robots 20.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”; Fig. 13 and [0137]: “As shown in FIG. 13, a planned moving track in future of the mobile robot 201 is displayed on the display section 43 as a line L3. By displaying such a line L3, the user U can intuitively grasp a moving destination of the mobile robot 201. Consequently, since teaching work adapted to the moving destination can be performed, it is possible to achieve efficiency of the teaching work.”; [0138]: “About the mobile robots 202 to 206 other than the mobile robot 201 selected by the selection operation, moving tracks may be displayed on the display section 43 as lines.”).
Wang fails to explicitly teach displaying, as an animation, the motion using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion using a three-dimensional model (see at least Fig. 3 and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103. For instance, a cursor or a vertical bar is displayed at a position on a time base on the event display area 108 corresponding to a drawn frame being displayed on the 3D model display area 107 to associate these two displays with each other. Still further, in a case where the animation (moving image) display is made on the 3D model display area 107, the display of the event display area 108 is automatically scrolled such that the position on the time base corresponding to the display of the 3D model display area 107 is accommodated within the display area. In the same time, the cursor and the vertical bar described above are automatically moved such they assume the positions on the time bar corresponding to the display of the 3D model display area 107.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to display, as an animation, motion using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of a robot for the user to spatiality visualize.
Regarding claim 10, modified Wang teaches the limitations of claim 1. Wang further teaches further comprising:
a connection terminal to which the operating device is configured to be connected via wire (see at least [0026]: “The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0080]: “Further, examples of the external interface shown in FIG. 5 include various connectors for communication. Examples of the connectors for communication include a USB (Universal Serial Bus) connector, an RS-232C connector, and a wired LAN (Local Area Network). The external interface includes a transceiver capable of performing wireless communication by various wireless communication standards such as a wireless LAN, a Bluetooth (registered trademark), and specific low-power wireless communication.”,
wherein a state where the operating device is connected to the information processing apparatus includes a state where the operating device is connected to the connection terminal via wire (see at least [0080] Further, examples of the external interface shown in FIG. 5 include various connectors for communication. Examples of the connectors for communication include a USB (Universal Serial Bus) connector, an RS-232C connector, and a wired LAN (Local Area Network). The external interface includes a transceiver capable of performing wireless communication by various wireless communication standards such as a wireless LAN, a Bluetooth (registered trademark), and specific low-power wireless communication.”; [0127]: “FIG. 12 is a diagram showing the display section 43 representing a communication state between the mobile robot 206 and the portable teaching device 40.”; [0144]: “Specifically, in FIG. 14, a state in which a real image of the mobile robots 201 to 203 is seen via the lens 81 is shown. The display content 85 corresponding to the mobile robots 201 to 203 is displayed as, for example, character information to be superimposed on the real image. Examples of the display content 85 include, besides characters, an image and a moving image. Examples of the character information include all kinds of information concerning the mobile robots 201 to 203 such as a model name, possibility of connection, and a name of work being executed.”).
Regarding claim 11, modified Wang teaches the limitations of claim 1. Wang further teaches further comprising:
a communication portion configured to perform wireless communication with the operating device (see at least [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0082]: “For example, depending on the configuration of the first detecting section 44, the portable teaching device 40 may include a wireless antenna, an RFID tag, a beacon receiver, an ultrasonic transducer, a photodiode, a laser diode, a camera, and a terrestrial magnetism sensor.”),
wherein a state where the operating device has been connected to the information processing apparatus includes a state where the communication portion has established the wireless communication with the operating device (see at least [0041]: “The teaching-device control section 46 selects one mobile robot 20 based on the input operation received by the input receiving section 45 and causes the communication section 41 to establish communication with the mobile robot 20. The teaching-device control section 46 outputs various operation signals to the mobile robot 20 with which the communication is established. Consequently, the teaching-device control section 46 performs teaching work on the mobile robot 20. The teaching-device control section 46 changes a lighting pattern of the indicator lamp section 47 according to a communication state in the communication section 41.”).
Regarding claim 12, Wang teaches an operating device (see at least Fig. 2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”) comprising:
an information processing portion (see at least Figs. 1-2 and [0027]: “The portable teaching device 40 shown in FIG. 2 includes a communication section 41, a display control section 42, a display section 43, a first detecting section 44, an input receiving section 45, a teaching-device control section 46, and an indicator lamp section 47.”; [0077]: “The portable teaching device 40 shown in FIG. 5 includes a processor, a memory, an external interface, an input device, and an output device communicably coupled to one another.”); and
a display portion (see at least Figs. 1-2 and [0027]: “The portable teaching device 40 shown in FIG. 2 includes a communication section 41, a display control section 42, a display section 43, a first detecting section 44, an input receiving section 45, a teaching-device control section 46, and an indicator lamp section 47.”),
wherein the information processing portion is configured to cause the display portion to display, based on first information, a motion of a first robot using a model corresponding to the first robot, after the operating device is connected to an information processing apparatus and the first information is sent from the information processing apparatus to the operating device (see at least Figs. 2-3, 6-9, and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on the motion of the first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”).
Wang fails to explicitly teach displaying, as an animation, a motion of a robot using a three-dimensional model corresponding to the robot.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, a motion of a robot using a three-dimensional model corresponding to the robot (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to display, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 14, modified Wang teaches the limitations of claim 12. Wang further teaches wherein the information processing portion is configured to cause the display portion to display a model corresponding to an object around the first robot (see at least Figs. 2-3, 6-14, and [0038]: “Examples of a detection method for the marker M1 by a marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the portable teaching device 40 to the marker M1 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0061]: “Examples of a detection method for the marker M2 by the marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the mobile robot 20 to the marker M2 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0070]: “Any other equipment or the like may be provided in the mobile robot 20. Specifically, examples of the equipment include an imaging section that images a work target object, the mobile robot 20, or the periphery of the mobile robot 20 and various sensors such as a force sensor that detects an external force applied to the robot arm 60.”).
Wang fails to explicitly teach displaying a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays a three-dimensional model (see at least Fig. 3 and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to display a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation for the user to spatiality visualize.
Regarding claim 15, modified Wang teaches the limitations of claim 12. Wang further teaches wherein the information processing portion is configured to cause the display portion to further display a captured image obtained by an image capture apparatus capturing an image, the image capture apparatus being disposed on the first robot (see at least Figs. 2-3, 6-14, and [0038]: “Examples of a detection method for the marker M1 by a marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the portable teaching device 40 to the marker M1 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0061]: “Examples of a detection method for the marker M2 by the marker detecting section 72 include detection by vision for detecting the marker M1 using a camera. A distance and a direction from the mobile robot 20 to the marker M2 can be calculated based on an image captured by the camera. The detection by vision can be substituted by, for example, a method by ultrasound or a distance measuring method by laser.”; [0063]: “With such a second detecting section 70, it is possible to measure the present position of the mobile robot 20 even with a relatively simple configuration. Accordingly, it is possible to achieve a reduction in the cost and a reduction in the size of the second detecting section 70.”; [0070]: “Any other equipment or the like may be provided in the mobile robot 20. Specifically, examples of the equipment include an imaging section that images a work target object, the mobile robot 20, or the periphery of the mobile robot 20 and various sensors such as a force sensor that detects an external force applied to the robot arm 60.”).
Regarding claim 18, modified Wang teaches the limitations of claim 12. Wang further teaches wherein the information processing portion is configured to display the first image and a second image on the display portion if the information processing portion receives the first information and second information from the information processing apparatus (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on the motion of the first robot operated by the operating device, the second information being information on a motion of a second robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”), the first image being an image that corresponds to the first information, the second image being an image that corresponds to the second information (see at least [0025]: “The robot system 1 may include one mobile robot 20. However, in FIG. 1, the robot system 1 includes a plurality of mobile robots 20.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”; Fig. 13 and [0137]: “As shown in FIG. 13, a planned moving track in future of the mobile robot 201 is displayed on the display section 43 as a line L3. By displaying such a line L3, the user U can intuitively grasp a moving destination of the mobile robot 201. Consequently, since teaching work adapted to the moving destination can be performed, it is possible to achieve efficiency of the teaching work.”; [0138]: “About the mobile robots 202 to 206 other than the mobile robot 201 selected by the selection operation, moving tracks may be displayed on the display section 43 as lines.”).
Regarding claim 21, Wang teaches a robot system (see at least Fig. 1) comprising:
a first robot (see at least Figs. 1-2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”); and
the operating device (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20.”); and
an information processing apparatus (see at least Fig. 2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”) having an information processing portion (see at least Figs. 1-2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”; [0044]: “The mobile robot 20 shown in FIG. 2 includes the truck 50, the robot arm 60, a communication section 22, and a second detecting section 70.”; [0078]: “Examples of the processor shown in FIG. 5 include a CPU (Central Processing Unit).”),
wherein the information processing portion is configured to send first information to the operating device if the operating device is connected to the information processing apparatus (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on a motion of the first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”), and
wherein the information processing portion is configured to cause a display portion included in the operating device to display, based on the first information, the motion of the first robot using a model corresponding to the first robot, after the operating device is connected to the information processing apparatus and the first information is sent from the information processing apparatus to the operating device (see at least Figs. 6-9 and [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”).
Wang fails to explicitly teach displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to displays, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 22, modified Wang teaches the limitations of claim 21. Wang further teaches the robot system comprising:
at least two of the first robot (see at least Fig. 1); and
at least two of the information processing apparatus that correspond to the at least two of the first robot (see at least Figs. 1-2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”),
wherein if the operating device is connected to a predetermined information processing apparatus of the at least two of the information processing apparatus, the predetermined information processing apparatus sends the first information to the operating device (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on the motion of the first robot that corresponds to the predetermined information processing apparatus (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”).
Regarding claim 23, Wang teaches an information processing method performed by an information processing portion, the method comprising:
sending, by the information processing portion, first information to an operating device if the operating device is connected to the information processing apparatus (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on a motion of a first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.” The examiner would like to point out that the phrase “sending, by the information processing portion, first information to an operating device if the operating device is connected…” is a conditional limitation. This means that the functionality or operation of the invention is dependent on certain conditions or circumstances. In this case, sending, by the information processing portion, first information is done if the operating device is connected. Broadly interpreted, if the condition for performing a contingent step is not satisfied, the performance recited by the step need not be carried out in order for the claimed method to be perform. The examiner recommends updating the phrase to make it a required limitation.) and
causing, by the information processing portion, a display portion included in the operating device to display, based on the first information, the motion of the first robot using a model corresponding to the first robot, after the operating device is connected to the information processing apparatus and the first information is sent from the information processing apparatus to the operating device (see at least Figs. 6-9 and [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”).
Wang fails to explicitly teach displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to displays, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 24, Wang teaches a control method of an operating device including an information processing portion and a display portion (see at least [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20.”), the method comprising:
causing, by the information processing portion, the display portion to display, based on first information, a motion of a first robot using a model corresponding to the first robot, after the operating device is connected to an information processing apparatus and the first information is sent from the information processing apparatus to the operating device (see at least Figs. 2-3, 6-14, and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on the motion of the first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”)
Wang fails to explicitly teach displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to displays, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 25, Wang teaches a method using a robot system including a first robot (see at least Figs. 1-2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”), an operating device (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20.”), and
an information processing apparatus (see at least Fig. 2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”) having an information processing portion (see at least Figs. 1-2 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”; [0044]: “The mobile robot 20 shown in FIG. 2 includes the truck 50, the robot arm 60, a communication section 22, and a second detecting section 70.”; [0078]: “Examples of the processor shown in FIG. 5 include a CPU (Central Processing Unit).”), the method comprising:
sending, by the information processing portion, first information to an operating device if the operating device is connected to the information processing apparatus (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on a motion of a first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.” The examiner would like to point out that the phrase “sending, by the information processing portion, first information to an operating device if the operating device is connected…” is a conditional limitation. This means that the functionality or operation of the invention is dependent on certain conditions or circumstances. In this case, sending, by the information processing portion, first information is done if the operating device is connected. Broadly interpreted, if the condition for performing a contingent step is not satisfied, the performance recited by the step need not be carried out in order for the claimed method to be perform. The examiner recommends updating the phrase to make it a required limitation.), and
causing, by the information processing portion, a display portion included in the operating device to display, based on the first information, the motion of the first robot using a model corresponding to the first robot, after the operating device is connected to the information processing apparatus and the first information is sent from the information processing apparatus to the operating device (see at least Figs. 2-3, 6-14, and [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”).
Wang fails to explicitly teach a method of manufacturing products by using the robot system and displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that a method of manufacturing products by using a robot system (see at least Fig. 13 and [0145]: “This configuration makes it possible to operate the robotic system 1001 as a production unit composing a production system (production line) based on the robot control data optimized by the abovementioned process and to manufacture articles by the robotic system 1001.”) and displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to manufacture products by using the robot system and displays, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to manufacture articles from the robot system and provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 26, Wang teaches a non-transitory computer-readable recording medium storing a program that causes a computer to execute an information processing method performed by an information processing portion (see at least Fig. 5 and [0073]: “The robot control section 31 reads out and executes a program stored in the storing section 32 to thereby control driving of the robot arm 60 and cause the robot arm 60 to perform work. Selection of a program is performed based on an operation signal from the portable teaching device 40.”; [0078]: “Examples of the processor shown in FIG. 5 include a CPU (Central Processing Unit).”; [0079]: “Examples of the memory shown in FIG. 5 include a volatile memory such as a RAM (Random Access Memory) and a nonvolatile memory such as a ROM (Read Only Memory). The memory is not limited to an undetachable type and may be a detachable external storage device.”), the method comprising:
sending, by the information processing portion, first information to an operating device if the operating device is connected to the information processing apparatus (see at least Fig. 2 and [0026]: “The portable teaching device 40 is a device that teaches the mobile robots 20. The portable teaching device 40 has portability for enabling the portable teaching device 40 to move even in a state in which a user holds the portable teaching device 40. The portable teaching device 40 and the mobile robots 20 are coupled by wireless communication in the figures but may be coupled by wired communication.”; [0028]: “The communication section 41 includes a transceiver configured to couple the portable teaching device 40 and the mobile robots 20 by wireless communication or wired communication.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0029]: “When communication between the communication section 41 and a specific mobile robot 20 is established, the display control section 42 differentiates display of the mobile robot 20 from display of the other mobile robots 20 to thereby visually display a communication state.”; [0071]: “When the robot system 1 includes a plurality of mobile robots 20, the configurations of the mobile robots 20 are the same as the configuration explained above. The mobile robots 20 have peculiar IDs (identification signs). In communication between the portable teaching device 40 and the mobile robots 20, it is possible to establish communication between a specific mobile robot 20 and the portable teaching device 40 by transmitting and receiving data to which the IDs are given.”), the first information being information on a motion of a first robot operated by the operating device (see at least Figs. 6-9 and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.” The examiner would like to point out that the phrase “sending, by the information processing portion, first information to an operating device if the operating device is connected…” is a conditional limitation. This means that the functionality or operation of the invention is dependent on certain conditions or circumstances. In this case, sending first information is done if the operating device is connected. Broadly interpreted, if the condition for performing a contingent step is not satisfied, the performance recited by the step need not be carried out in order for the claimed method to be perform. The examiner recommends updating the phrase to make it a required limitation.), and
causing, by the information processing portion, a display portion included in the operating device to display, based on the first information, the motion of the first robot using a model corresponding to the first robot, after the operating device is connected to the information processing apparatus and the first information is sent from the information processing apparatus to the operating device (see at least Figs. 2-3, 6-14, and [0098]: “In FIG. 6, the six mobile robots 20 are located to surround the user U who is holding the portable teaching device 40. At this time, present positions of the mobile robots 20 and a present position of the user U shown in FIG. 6 are displayed on the display section 43 of the portable teaching device 40. Therefore, as shown in FIG. 7, icons 20A indicating the six mobile robots 20 and an icon 40A indicating the portable teaching device 40 are displayed in disposition corresponding to the direction of the portable teaching device 40.”; [0104]: “FIG. 8 is a diagram showing an example of a track T1 of movement of one mobile robot 201. FIG. 9 is a diagram showing an example in which the track T1 of the mobile robot 201 shown in FIG. 8 is displayed on the display section 43 as a line L1.”).
Wang fails to explicitly teach displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Sugaya and provide a means to displays, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 28, modified Wang teaches the limitations of claim 1. Wang further teaches wherein the information processing apparatus does not move autonomously (see at least Fig. 1 and [0023]: “A robot system 1 shown in FIG. 1 includes mobile robots 20, control devices 30, and a portable teaching device 40.”; [0026] The portable teaching device 40 is a device that teaches the mobile robots 20.”; [0051]: “The truck 50 is desirably a truck that moves with power but may be a truck that is manually moved.” Wang teaches a portable teaching device is utilized by a user to manually teach the mobile robot and communicates with the control device in order to control the robot. Broadly interpreted, the portable teaching device and the control device (controllers) does not move autonomously as it does not have any actuators for movement.).
Claim Rejections - 35 USC § 103
8. Claims 6 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 20200376680, hereinafter Wang) and Sugaya (US 20190221037, hereinafter Sugaya) in view of Guilbert (US 9919427, hereinafter Guilbert).
Regarding claim 6, modified Wang teaches the limitations of claim 1. Wang further teaches a storage portion (see at least Fig. 2 and [0072]: “The control device 30 includes a robot control section 31, a storing section 32, and a route acquiring section 33.”),
wherein the storage portion is configured to store movement data as the first information, (see at least [0073]: “The robot control section 31 reads out and executes a program stored in the storing section 32 to thereby control driving of the robot arm 60 and cause the robot arm 60 to perform work. Selection of a program is performed based on an operation signal from the portable teaching device 40.”; [0074]: “The route acquiring section 33 has a function of acquiring a planned moving track of the truck 50. The moving track is determined based on the present position of the mobile robot 20 and the program stored in the storing section 32.”), and
wherein the information processing portion is configured to cause the display portion to display, based on the movement data, the motion of the first robot corresponding to the first robot (see at least Figs. 2-3, 6-14, and [0103]: “The positions of the icons 201A to 206A and the position of the icon 40A in the display section 43 are sequentially updated according to the present positions of the mobile robots 201 to 206 and the present position of the portable teaching device 40. Accordingly, when the mobile robots 201 to 206 move or the user U holding the portable teaching device 40 moves, the movement is reflected on display content of the display section 43 according to the movement.”; [0105]: “When the mobile robot 201 moves in a route indicated by the track T1 as shown in FIG. 8, the icon 201A corresponding to a present position of the mobile robot 201 at a moving destination and the line L1 corresponding to the track T1 are displayed in FIG. 9. Consequently, the user U can intuitively confirm whether the mobile robot 201 has moved in a predetermined route. When the portable teaching device 40 moves in a route indicated by a track T2 as shown in FIG. 8, the icon 40A corresponding to a present position of the portable teaching device 40 at a moving destination and a line L2 corresponding to the track T2 are displayed in FIG. 9. Consequently, the user U can intuitively confirm in which route the user U gripping the portable teaching device 40 has moved.”).
Wang fails to explicitly teach movement data as trajectory data.
However, Guilbert teaches a method and system for visualizing robot trajectory points that represent movement data as trajectory data and sends image data that represents a motion of a first robot based on the trajectory data (see at least Figs. 4A-4B and Col. 4, lines 3-17: “Trajectory points and/or other information may be displayed to a user within an augmented reality presentation in various ways. In some examples, full trajectories may overlaid onto the image of the robot's environment (e.g., as paths made up of sequences of connected trajectory points). In additional examples, the display may include a virtual simulation of the robot's movement through a given trajectory or sequence of trajectory points. For instance, a virtual simulation may graphically illustrate how the robot motion will actually look within a particular workcell before the robot is commanded to physically move within the workcell. In another example, one or more collision points may also be displayed which warn the user of possible collisions that may occur if the robot is commanded to move through a given trajectory in a particular workcell.”; Col. 9, lines 36-41: “FIG. 4D illustrates an image of a robot's environment with overlaid virtual trajectory points, according to an example embodiment. More specifically, the display interface 450 may show an image of the workcell 400 of robot 402 along with virtual trajectory points 452, 454, 456, 458, 460, and 462 graphically overlaid onto the image.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Guilbert and provide a means to represent movement data as trajectory data and sends image data that represents a motion of a first robot based on the trajectory data, with a reasonable expectation of success, in order to illustrate a more detailed representation of the motion of the robot on the image.
The combination of Wang and Guilbert fails to explicitly teach displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang and Guilbert to incorporate the teachings of Sugaya and provide a means to displays, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Regarding claim 16, modified Wang teaches the limitations of claim 12. Wang further teaches wherein the first information is movement data of the first robot, and wherein the information processing portion is configured to cause the display portion to display, based on the movement data, the motion of the first robot corresponding to the first robot (see at least Figs. 2-3, 6-14, and [0103]: “The positions of the icons 201A to 206A and the position of the icon 40A in the display section 43 are sequentially updated according to the present positions of the mobile robots 201 to 206 and the present position of the portable teaching device 40. Accordingly, when the mobile robots 201 to 206 move or the user U holding the portable teaching device 40 moves, the movement is reflected on display content of the display section 43 according to the movement.”; [0105]: “When the mobile robot 201 moves in a route indicated by the track T1 as shown in FIG. 8, the icon 201A corresponding to a present position of the mobile robot 201 at a moving destination and the line L1 corresponding to the track T1 are displayed in FIG. 9. Consequently, the user U can intuitively confirm whether the mobile robot 201 has moved in a predetermined route. When the portable teaching device 40 moves in a route indicated by a track T2 as shown in FIG. 8, the icon 40A corresponding to a present position of the portable teaching device 40 at a moving destination and a line L2 corresponding to the track T2 are displayed in FIG. 9. Consequently, the user U can intuitively confirm in which route the user U gripping the portable teaching device 40 has moved.”).
Wang fails to explicitly teach movement data as trajectory data.
However, Guilbert teaches a method and system for visualizing robot trajectory points that comprises an image that represents a motion of a first robot based on trajectory data (see at least Figs. 4A-4B and Col. 4, lines 3-17: “Trajectory points and/or other information may be displayed to a user within an augmented reality presentation in various ways. In some examples, full trajectories may overlaid onto the image of the robot's environment (e.g., as paths made up of sequences of connected trajectory points). In additional examples, the display may include a virtual simulation of the robot's movement through a given trajectory or sequence of trajectory points. For instance, a virtual simulation may graphically illustrate how the robot motion will actually look within a particular workcell before the robot is commanded to physically move within the workcell. In another example, one or more collision points may also be displayed which warn the user of possible collisions that may occur if the robot is commanded to move through a given trajectory in a particular workcell.”; Col. 9, lines 36-41: “FIG. 4D illustrates an image of a robot's environment with overlaid virtual trajectory points, according to an example embodiment. More specifically, the display interface 450 may show an image of the workcell 400 of robot 402 along with virtual trajectory points 452, 454, 456, 458, 460, and 462 graphically overlaid onto the image.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Guilbert and provide an image that represents a motion of a first robot based on trajectory data, with a reasonable expectation of success, in order to illustrate a more detailed representation of the motion of the robot on the image.
The combination of Wang and Guilbert fails to explicitly teach displaying, as an animation, the motion of the robot using a three-dimensional model.
However, Sugaya teaches an apparatus and method for outputting display information of a robotic system that displays, as an animation, motion of a robot using a three-dimensional model (see at least Figs. 3, 12, and [0039]: “The displays of the 3D model display area 107, the event display area 108 and the analysis condition setting area 109 are controlled so as to be linked through the operation of the 3D model of the robotic system within the virtual environment and through an operation of the user made through the display unit 102 and the operating unit 103.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang and Guilbert to incorporate the teachings of Sugaya and provide a means to displays, as an animation, motion of a robot using a three-dimensional model, with a reasonable expectation of success, in order to provide a 3D representation of the robot for the user to spatiality visualize.
Claim Rejections - 35 USC § 103
9. Claim 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 20200376680, hereinafter Wang) and Sugaya (US 20190221037, hereinafter Sugaya) in view of Tarui (US 20190030715, hereinafter Tarui).
Regarding claim 27, modified Wang teaches the limitations of claim 1. Wang further teaches wherein the first information is a robot program used by the information processing portion when controlling the first robot before the operating device is connected to the information processing apparatus (see at least [0073]: “The robot control section 31 reads out and executes a program stored in the storing section 32 to thereby control driving of the robot arm 60 and cause the robot arm 60 to perform work. Selection of a program is performed based on an operation signal from the portable teaching device 40.”; [0079]: “Examples of the memory shown in FIG. 5 include a volatile memory such as a RAM (Random Access Memory) and a nonvolatile memory such as a ROM (Read Only Memory). The memory is not limited to an undetachable type and may be a detachable external storage device.”; [0080]: “Further, examples of the external interface shown in FIG. 5 include various connectors for communication. Examples of the connectors for communication include a USB (Universal Serial Bus) connector, an RS-232C connector, and a wired LAN (Local Area Network). The external interface includes a transceiver capable of performing wireless communication by various wireless communication standards such as a wireless LAN, a Bluetooth (registered trademark), and specific low-power wireless communication.”).
Wang fails to explicitly teach wherein the information is a robot program last used when controlling the robot before the operating device is connected.
However, Tarui teaches an apparatus and system for a robot controller that comprises information that is a robot program last used when controlling a robot before an operating device is connected (see at least [0054]: “In the program storage unit 430, the setup programs 450 corresponding to the individual teaching positions and the operation program 460 of the robot 300 are previously stored…Then, when the start of the setup programs 450 corresponding to all the teaching positions is completed, the robot controller 400 can be made to execute the operation program 460.”).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Tarui and provide information that is a robot program last used when controlling a robot before an operating device is connected, with a reasonable expectation of success, in order to execute a program that was previously saved to allow the system to continue where it left off.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIEN MINH LE whose telephone number is (571)272-3903. The examiner can normally be reached Monday to Friday (8:30am-5:30pm eastern time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi Tran can be reached on (571)272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.M.L./
Examiner, Art Unit 3656
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights