Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This communication is responsive to Application No. 18/821,693 and the amendments filed on 4/15/2026.
3. Claims 1 and 3-16 are presented for examination.
Information Disclosure Statement
4. The information disclosure statements (IDS) submitted on 8/30/2024 and 9/4/2025
have been fully considered by the Examiner.
Response to Arguments
5. Applicant’s arguments with respect to the rejection of claim(s) 1-15 under 35 U.S.C. 102(a)(1) and/or 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Regarding independent claim 1, the Examiner agrees that US 20220182533 A1 to Ino fails to teach all of the amended limitations of the claim. However, in light of the amendments and the Applicant’s remarks, an updated search was conducted, and a new ground concerning claim 1 has been determined, in which will be described later.
Regarding dependent claim 2, this claim has been cancelled, and thus, is withdrawn from further consideration.
Regarding dependent claims 3-13, as all of these claims depend from claim 1, are still rejected, in which will be described later.
Regarding independent claims 14 and 15, as these claims contain similar limitations as claim 1, are still rejected for similar reasons as claim 1 is, in which will be described later.
Claim Rejections - 35 USC § 103
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claim(s) 1, 3, 8, 9, 14, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oono et al. (US 20200306957 A1 hereinafter Oono) in view of Hayashi (US 20210283516 A1 hereinafter Hayashi).
Regarding Claim 1, Oono teaches an imaging system comprising: a camera ([0029] via “The robot apparatus 5 in the present embodiment includes a camera 7 disposed so as to capture an image of at least one of the robot 1 and the hand 2.”); and
at least one processor ([0051] via “The robot controller 4 includes a signal processing unit 31 ….”), wherein the at least one processor is configured to execute processes ([0051] via “The robot controller 4 includes a signal processing unit 31 that receives signals from the position detectors 18 and a signal from the camera 7 and processes the signals.”) comprising:
in a case in which (i) a gesture that a robot is to be caused to execute ([0036] via “The operation control unit 43 sends an operation command for driving the robot 1 to the robot drive unit 45 based on the operation program 41.”), (Note: The Examiner interprets the operation command of Oono as the gesture.) at a time of video imaging by the camera ([0036] via “Further, the operation control unit 43 sends a command for capturing an image to the camera 7 based on the operation program 41.”), ([0046] via “In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1.”) is selected ([0034] via “An operation program 41, which has previously been created in order to control the robot 1, the hand 2, and the conveyor 8, is input to the robot controller 4.”), ([0036] via “The operation control unit 43 sends an operation command for driving the robot 1 to the robot drive unit 45 based on the operation program 41.”), and
(ii) the video imaging by the camera is to be started with the robot as a subject of the video imaging ([0058] via “The camera 7 captures images of the robot 1 and the hand 2 at a time interval that coincides with the control cycle.”),
controlling the camera to perform the video imaging of the robot ([0058] via “The camera 7 captures images of the robot 1 and the hand 2 at a time interval that coincides with the control cycle.”) so that the video imaging of the robot ends as a timing corresponding to a timing at which the robot ends the gesture ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4
shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot ends, so too does the image capturing.); and
in response to the gesture that the robot is to be caused to execute being selected [from among the plurality of predetermined gestures], transmitting a signal instructing an execution start of the gesture to the robot in conjunction with a start of the video imaging of the robot ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot starts, so too does the image capturing.).
Oono is silent on selecting the gesture from among a plurality of predetermined gestures.
However, Hayashi teaches selecting the gesture from among a plurality of predetermined gestures ([0071] via “The operation control unit 222 selects a motion of the robot 100 from a multiple of motions in the motion storage unit 232. A selection probability is correlated to each motion for each situation”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Hayashi of selecting the gesture from among a plurality of predetermined gestures. Doing so causes the robot to perform the most optimal gesture based on the current situation, as stated by Hayashi ([0071] via “The operation control unit 222 selects a motion of the robot 100 from a multiple of motions in the motion storage unit 232. A selection probability is correlated to each motion for each situation. For example, a selection method such that a motion A is executed with a 20% probability when a pleasant action is performed by an owner, and a motion B is executed with a 5% probability when a temperature reaches 30 degrees or higher, is defined.”). The Examiner notes that Oono also comprises a memory and is able to automatically perform a gesture, as stated in paragraph [0034] of Oono, and is capable of incorporating such a feature.
Regarding Claim 3, modified reference Oono teaches the imaging system according to claim 1, further comprising the robot ([0023] via “The robot 1 ….”),
wherein the robot starts execution of the gesture upon receiving the signal instructing the execution start of the gesture ([0036] via “The operation control unit 43 sends an operation command for driving the robot 1 to the robot drive unit 45 based on the operation program 41. The robot drive unit 45 includes an electric circuit that drives the robot drive motors 22. The robot drive unit 45 supplies electricity to the robot drive motors 22 based on the operation command.”).
Regarding Claim 8, modified reference Oono teaches the imaging system according to claim 1, wherein the processes further comprise, in response to the gesture that the robot is to be caused to execute being selected [from among the plurality of predetermined gestures], controlling the video imaging of the robot by sending a signal instructing a start of the video imaging of the robot to the camera in conjunction with the start of the gesture by the robot ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), ([0058] via “At step 72, the operation control unit 43 performs an imaging step for capturing an image of an area where the workpiece W is placed on the conveyor 8 by using the camera 7. The camera 7 captures images of the robot 1 and the hand 2 at a time interval that coincides with the control cycle.”).
Regarding Claim 9, modified reference Oono teaches the imaging system according to claim 8, wherein the camera starts the video imaging of the robot upon receiving the signal instructing the start of the video imaging of the robot ([0046] via “The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1
to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), ([0058] via “At step 72, the operation control unit 43
performs an imaging step for capturing an image of an area where the workpiece W is placed on the conveyor 8 by using the camera 7. The camera 7 captures images of the robot 1 and the hand 2 at a time interval that coincides with the control cycle.”).
Regarding Claim 14, Oono teaches an imaging method, comprising: selecting a gesture that a robot is to be caused to execute ([0034] via “An operation program 41, which has previously been created in order to control the robot 1, the hand 2, and the conveyor 8, is input to the robot controller 4.”), ([0036] via “The operation control unit 43 sends an operation command for driving the robot 1 to the robot drive unit 45 based on the operation program 41.”), (Note: The Examiner interprets the operation command of Oono as the gesture.) at a time of video imaging by a camera ([0036] via “Further, the operation control unit 43 sends a command for capturing an image to the camera 7 based on the operation program 41.”), ([0046] via “In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1.”), in a case in which the video imaging by the camera is to be started with the robot as a subject of the video imaging ([0058] via “The camera 7 captures images of the robot 1 and the hand 2 at a time interval that coincides with the control cycle.”); and
in response to the gesture that the robot is to be caused to execute being selected [from among the plurality of predetermined gestures]: transmitting a signal instructing a start of execution by the robot of the selected gesture in conjunction with a start of the video imaging of the robot ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot starts, so too does the image capturing.); and
controlling the camera to end the video imaging of the robot at a timing corresponding to a timing at which the robot ends the gesture ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot ends, so too does the image capturing.).
Oono is silent on selecting the gesture from among a plurality of predetermined gestures.
However, Hayashi teaches selecting the gesture from among a plurality of predetermined gestures ([0071] via “The operation control unit 222 selects a motion of the robot 100 from a multiple of motions in the motion storage unit 232. A selection probability is correlated to each motion for each situation”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Hayashi selecting the gesture from among a plurality of predetermined gestures. Doing so causes the robot to perform the most optimal gesture based on the current situation, as stated by Hayashi ([0071] via “The operation control unit 222 selects a motion of the robot 100 from a multiple of motions in the motion storage unit 232. A selection probability is correlated to each motion for each situation. For example, a selection method such that a motion A is executed with a 20% probability when a pleasant action is performed by an owner, and a motion B is executed with a 5% probability when a temperature reaches 30 degrees or higher, is defined.”). The Examiner notes that Oono also comprises a memory and is able to automatically perform a gesture, as stated in paragraph [0034] of Oono, and is capable of incorporating such a feature.
Regarding Claim 15, Oono teaches a non-transitory storage medium storing a program ([0034] via “The robot controller 4 includes a storage unit 42 that stores information related to control of the robot 1, the hand 2, and the conveyor 8. The storage unit 42 can be configured by a storage medium capable of storing information, such as a volatile memory, a nonvolatile memory, or a hard disk. The operation program 41 is stored in the storage unit
42.”) readable by a computer of an imaging system ([0035] via “The processor functions as the operation control unit 43 by reading the operation program 41 and performing a control operation defined in the operation program 41.”), the program causing the computer to execute processes comprising:
selecting a gesture that a robot is to be caused to execute ([0034] via “An operation program 41, which has previously been created in order to control the robot 1, the hand 2, and the conveyor 8, is input to the robot controller 4.”), ([0036] via “The operation control unit 43 sends an operation command for driving the robot 1 to the robot drive unit 45 based on the operation program 41.”), (Note: The Examiner interprets the operation command of Oono as the gesture.) at a time of video imaging by a camera ([0036] via “Further, the operation control unit 43 sends a command for capturing an image to the camera 7 based on the operation program 41.”), ([0046] via “In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1.”), in a case in which the video imaging by the camera is to be started with the robot as a subject of the video imaging ([0058] via “The camera 7 captures images of the robot 1 and the hand 2 at a time interval that coincides with the control cycle.”); and
in response to the gesture that the robot is to be caused to execute being selected [from among the plurality of predetermined gestures]: transmitting a signal instructing a start of execution by the robot of the selected gesture in conjunction with a start of the video imaging of the robot ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images
51a to 51f captured at time t1 to time t6. The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot starts, so too does the image capturing.); and
controlling the camera to end the video imaging of the robot at a timing corresponding to a timing at which the robot ends the gesture ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot ends, so too does the image capturing.).
Oono is silent on selecting the gesture from among a plurality of predetermined gestures.
However, Hayashi teaches selecting the gesture from among a plurality of predetermined gestures ([0071] via “The operation control unit 222 selects a motion of the robot 100 from a multiple of motions in the motion storage unit 232. A selection probability is correlated to each motion for each situation”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Hayashi of selecting the gesture from among a plurality of predetermined gestures. Doing so causes the robot to perform the most optimal gesture based on the current situation, as stated by Hayashi ([0071] via “The operation control unit 222 selects a motion of the robot 100 from a multiple of motions in the motion storage unit 232. A selection probability is correlated to each motion for each situation. For example, a selection method such that a motion A is executed with a 20% probability when a pleasant action is performed by an owner, and a motion B is executed with a 5% probability when a temperature reaches 30 degrees or higher, is defined.”). The Examiner notes that Oono also comprises a memory and is able to automatically perform a gesture, as stated in paragraph [0034] of Oono, and is capable of incorporating such a feature.
10. Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oono et al. (US 20200306957 A1 hereinafter Oono) in view of Hayashi (US 20210283516 A1 hereinafter Hayashi), and further in view of Fountain (US 20240185492 A1 hereinafter Fountain).
Regarding Claim 4, modified reference Oono teaches the imaging system according to claim 1, wherein the at least one processor ends the video imaging of the robot at the timing corresponding to the timing at which the robot ends the gesture ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images
51a to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1 to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot ends, so too does the image capturing.).
Oono is silent on by ending, based on an execution time length registered in advance in association with the gesture, the video imaging of the robot by the camera.
However, Fountain teaches by ending, based on an execution time length registered in advance in association with the gesture, the video imaging of the robot by the camera ([0046] via “For example, in the case where operation to move the position of the robot 3 is performed based on the movement operation data obtained by the operation data acquisition unit 131, the image data acquisition unit 133 initializes the capture image data stored in the memory unit 12. For example, the image data acquisition unit 133 deletes the capture image data stored in the memory unit 12 to initialize the capture image data.”), ([0047] via “Further, even if the robot 3 is present at the same position, after elapse of a long period of time, the environment around the robot 3 may change. … So, in the case where the operator U finishes operation of the robot 3, or after elapse of predetermined time, the image data acquisition unit 133 may initialize the capture image data stored in the memory unit 12.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Fountain wherein by ending, based on an execution time length registered in advance in association with the gesture, the video imaging of the robot by the camera. Doing so prevents the infiltration of unnecessary or irrelevant data from being captured after the robot already completes the gesture, as stated by Fountain ([0047] via “Further, even if the robot 3 is present at the same position, after elapse of a long period of time, the environment around the robot 3 may change. If a plurality of pieces of the past capture image data are combined even though the environment around the robot 3 has changed, mismatch occurs at the borders between the plurality of pieces of combined capture image data. So, in the case where the operator U finishes operation of the robot 3, or after elapse of predetermined time, the image data acquisition unit 133 may initialize the capture image data stored in the memory unit 12.”).
11. Claim(s) 5, 6, 10, and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oono et al. (US 20200306957 A1 hereinafter Oono) in view of Hayashi (US 20210283516 A1 hereinafter Hayashi), and further in view of Kalouche et al. (US 11794349 B2 hereinafter Kalouche).
Regarding Claim 5, modified reference Oono teaches the imaging system according to claim 1, wherein the at least one processor ends the video imaging of the robot at the timing corresponding to the timing at which the robot ends the gesture ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a
to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1
to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot ends, so too does the image capturing.).
Oono is silent on by ending the video imaging of the robot by the camera upon receiving, from the robot, a notification indicating that the gesture is ended.
However, Kalouche teaches by ending the video imaging of the robot by the camera upon receiving, from the robot, a notification indicating that the gesture is ended (Col. 8 lines 14-19, where “As shown in block 401, instructions may be captured by an imaging device, such as imaging device 254. The imaging device, such as camera 90, may be positioned in front of display device 16 of picking station 10 to capture the visual instructions sent by the WS, executed on the warehouse system 203 and displayed on the display device 16.”), (Col. 10 lines 10-26, where “By performing the pick and place functions generated from the visual instructions, the instructions in the visual instructions may be completed and completion of the instructions may be confirmed, as shown in block 409. Referring to FIG. 1, the picking station 10 may include physical completion buttons 62 and 64 that may be pressed when a visual instruction is completed. As described, one or more completion buttons may also be presented on display device 252, such as display 16, of warehouse system 203. By pressing a completion button the WS may be notified that the current visual instruction has been completed and the next visual instruction may be sent to the display 252. Alternatively, the “complete” or similar button on the display 252 may be selected to provide notification to the WS that the current instruction has been completed.”), (Note: See Figure 4 of Kalouche as well. The Examiner interprets that since the workflow has been determined to be complete, this also includes ending the capturing of the visual instructions of step 401.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kalouche wherein by ending the video imaging of the robot by the camera upon receiving, from the robot, a notification indicating that the gesture is ended. Doing so ends the process of the gesture performed by the robot such that the robot may either perform a next task or be done its tasks entirely, as stated above by Kalouche in Col. 10 lines 10-26.
Regarding Claim 6, modified reference Oono teaches the imaging system according to claim 5, further comprising the robot ([0023] via “The robot 1 ….”).
Oono is silent on wherein upon ending of the gesture, the robot sends the notification indicating that the gesture is ended.
However, Kalouche teaches wherein upon ending of the gesture, the robot sends the notification indicating that the gesture is ended (Col. 10 lines 10-26, where “By performing the pick and place functions generated from the visual instructions, the instructions in the visual instructions may be completed and completion of the instructions may be confirmed, as shown in block 409. Referring to FIG. 1, the picking station 10 may include physical completion buttons 62 and 64 that may be pressed when a visual instruction is completed. As described, one or more completion buttons may also be presented on display device 252, such as display 16, of warehouse system 203. By pressing a completion button the WS may be notified that the current visual instruction has been completed and the next visual instruction may be sent to the display 252. Alternatively, the “complete” or similar button on the display
252 may be selected to provide notification to the WS that the current instruction has been completed.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kalouche wherein upon ending of the gesture, the robot sends the notification indicating that the gesture is ended. Doing so ends the process of the gesture performed by the robot such that the robot may either perform a next task or be done its tasks entirely, as stated above by Kalouche.
Regarding Claim 10, modified reference Oono teaches the imaging system according to claim 8, but is silent on wherein the processes further comprise, in response to the robot ending the gesture, sending, to the camera, a notification indicating that the gesture has ended.
However, Kalouche teaches wherein the processes further comprise, in response to the robot ending the gesture, sending, to the camera, a notification indicating that the gesture has ended (Col. 8 lines 14-19, where “As shown in block 401, instructions may be captured by an imaging device, such as imaging device 254. The imaging device, such as camera 90, may be positioned in front of display device 16 of picking station 10 to capture the visual instructions sent by the WS, executed on the warehouse system 203 and displayed on the display device 16.”), (Col. 10 lines 10-26, where “By performing the pick and place functions generated from the visual instructions, the instructions in the visual instructions may be completed and completion of the instructions may be confirmed, as shown in block 409. Referring to FIG. 1, the picking station 10 may include physical completion buttons 62 and 64
that may be pressed when a visual instruction is completed. As described, one or more completion buttons may also be presented on display device 252, such as display 16, of warehouse system 203. By pressing a completion button the WS may be notified that the current visual instruction has been completed and the next visual instruction may be sent to the display 252. Alternatively, the “complete” or similar button on the display 252 may be selected to provide notification to the WS that the current instruction has been completed.”), (Note: See Figure 4 of Kalouche as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kalouche wherein the processes further comprise, in response to the robot ending the gesture, sending, to the camera, a notification indicating that the gesture has ended. Doing so ends the process of the gesture performed by the robot such that the robot may either perform a next task or be done its tasks entirely, as stated above by Kalouche in Col. 10 lines 10-26.
Regarding Claim 11, modified reference Oono teaches the imaging system according to claim 10, but is silent on wherein the camera ends the video imaging of the robot upon receiving the notification indicating that the gesture has ended.
However, Kalouche teaches wherein the camera ends the video imaging of the robot upon receiving the notification indicating that the gesture has ended (Col. 8 lines 14-19, where “As shown in block 401, instructions may be captured by an imaging device, such as imaging device 254. The imaging device, such as camera 90, may be positioned in front of display device 16 of picking station 10 to capture the visual instructions sent by the WS, executed on the warehouse system 203 and displayed on the display device 16.”), (Col. 10 lines 10-26, where “By performing the pick and place functions generated from the visual instructions, the instructions in the visual instructions may be completed and completion of the instructions may be confirmed, as shown in block 409. Referring to FIG. 1, the picking station 10 may include physical completion buttons 62 and 64 that may be pressed when a visual instruction is completed. As described, one or more completion buttons may also be presented on display device 252, such as display 16, of warehouse system 203. By pressing a completion button the WS may be notified that the current visual instruction has been completed and the next visual instruction may be sent to the display 252. Alternatively, the “complete” or similar button on the display 252 may be selected to provide notification to the WS that the current instruction has been completed.”), (Note: See Figure 4 of Kalouche as well. The Examiner interprets that since the workflow has been determined to be complete, this also includes ending the capturing of the visual instructions of step 401.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kalouche wherein the camera ends the video imaging of the robot upon receiving the notification indicating that the gesture has ended. Doing so ends the process of the gesture performed by the robot such that the robot may either perform a next task or be done its tasks entirely, as stated above by Kalouche in Col. 10 lines 10-26.
12. Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oono et al. (US 20200306957 A1 hereinafter Oono) in view of Hayashi (US 20210283516 A1 hereinafter Hayashi), and further in view of Sohmshetty et al. (US 20250085694 A1 hereinafter Sohmshetty) and Ino et al. (US 20220182533 A1 hereinafter Ino).
Regarding Claim 7, modified reference Oono teaches the imaging system according to claim 1, wherein: the at least one processor ends the video imaging of the robot at the timing corresponding to the timing at which the robot ends the gesture ([0045] via “Referring to FIGS. 1 and 2, the camera 7 in the present embodiment is formed so as to capture images at a time interval that coincides with the control cycle Δt of the robot 1. For example, the camera is formed so as to capture images at a time interval identical to the control cycle Δt. … Even in this case, the camera can capture images at a time interval identical to the control cycle.”), ([0046] via “FIG. 4 shows images 51a to 51f captured at time t1 to time t6. The images 51a
to 51f are images captured at a time interval Δt that coincides with the control cycle Δt. In other words, FIG. 4 shows images captured every time the operation control unit 43 sends an operation command to the robot 1 so as to cause the position and orientation of the robot 1
to change. The operation control unit 43 can send an imaging command to the camera 7 at the same time as the issuance of an operation command for the robot 1. In other words, the issuance of the operation command for the robot 1 and the issuance of the imaging command to the camera 7 can be synchronized.”), (Note: The Examiner interprets that since the captured images of Oono are synchronized to the control cycles of the robot, that when the control cycles of the robot ends, so too does the image capturing.).
Oono is silent on the processes further comprise determining, based on a video obtained by the video imaging of the robot, whether the robot has ended the gesture, and
by, in response to a determination that the robot has ended the gesture, ending the video imaging of the robot.
However, Sohmshetty teaches the processes further comprise determining, based on a video obtained by the video imaging of the robot, whether the robot has ended the gesture ([0025] via “The vision system 28 includes one or more cameras, such as a two-dimensional (2D) camera, a three-dimensional (3D) camera, a stereo vision camera, ….”), ([0044] via “During and/or after the finishing operation, the vision system 28 generates a finishing map based on updated scanned data and the sensing system 42 acquires real-time updated measurements of the die surface relating to the surface roughness of the target area and the geometry/contour of the die surface in step 96. The finishing map and surface roughness are compared with the previous image of the die surface or the CAD model to determine whether the finishing operation is complete or whether further finishing operation is required in step
98.”).
Further, Ino teaches by, in response to a determination that the robot has ended the gesture, ending the video imaging of the robot ([0082] via “The shutter 17 causes the captured image to be output every one second, in a case where one or more subjects are no longer detected by the image recognizer 12, the shutter 17 causes the outputting of the captured image by the camera 11 to end. Furthermore, in a case where one or more subjects are no longer detected by the image recognizer 12, the controller 13 causes the operation of the first camera robot 1 according to the operation pattern to end.”), ([0086] via “When the image recognizer 12 no longer detects one or more subjects, the controller 13 causes the communicator 19 to transmit, to the other camera robots (the second camera robot 2 and the third camera robot 3), operation end instruction information for ending operation of the other camera robots (the second camera robot 7 and the third camera robot 3). The communicator 19 transmits the operation end instruction information to the other camera robots (the second camera robot 2 and the third camera robot 3).”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Sohnshetty wherein the processes further comprise determining, based on a video obtained by the video imaging of the robot, whether the robot has ended the gesture. Doing so determines whether there are still additional gestures for the robot to perform in order to fully complete a task so as to not prematurely end the process, as stated by Sohmshetty ([0044] via “If the surface roughness of the target area is within the predetermined range of the target surface roughness, the method goes to step 102 to determine whether more target area(s) need to be finished in step 102. If no more target area needs to be finished, the method ends in step 104. If more target area(s) need to be finished, the method goes back to step 90 to select a desired finishing operation for another target area.”).
In addition, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Ino wherein by, in response to a determination that the robot has ended the gesture, ending the video imaging of the robot. As there is no longer any gesture significant to capture, the camera stops capturing video imaging, as stated above by Ino in both citations.
13. Claim(s) 12 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oono et al. (US 20200306957 A1 hereinafter Oono) in view of Hayashi (US 20210283516 A1 hereinafter Hayashi), and further in view of Sakamoto et al. (US 6505098 B1 hereinafter Sakamoto).
Regarding Claim 12, modified reference Oono teaches the imaging system according to claim 1, but is silent on wherein: the robot stores a setting of at least one of a personality parameter expressing a pseudo-personality or a growth parameter expressing pseudo-growth, and the gesture that the robot is to be caused to execute changes in accordance with at least one of the personality parameter or the growth parameter.
However, Sakamoto teaches wherein: the robot stores a setting of at least one of a personality parameter expressing a pseudo-personality or a growth parameter expressing pseudo-growth (Col. 24 lines 23-31, where “Concretely speaking, prepared for the pet robot 121 in this pet robot system 120 are four "growth steps" of "baby period," "child period," "young period" and "adult period." Preliminarily stored in a memory 122A (FIG. 19) of a controller 122 (FIG. 10) are action and motion models consisting of various kinds of control parameters and control programs to be used as bases of actions and motions related to four items of "walking condition," "motion," "action" and "sound" for each "growth step."”), and
the gesture that the robot is to be caused to execute changes in accordance with at least one of the personality parameter or the growth parameter (Col. 24 lines 51-58, where “When a total value of accumulative frequencies of the growth factors (hereinafter referred to as a total experience value of the growth factors) exceeds a predetermined threshold value, the controller 122 modifies the action and motion models for "baby period" into action and motion models for "child period" at a higher growth level (at which actions and motions are harder and more complicated) on the basis of the accumulative frequencies of the growth factors.”), (Col. 25 lines 15-23, where “As a result, the pet robot 121 changes stepwise "walking condition" from "tottering walk" to "firm walking," changes "motion" from "simple" to "upgraded and complicated," changes "action" from "monotonous" to "action with a purpose" and changes "sound" from "low and short" to "long and loud" as the pet robot 121 has ascended "growth step" (that is, "growth step" changes from "baby period" to "child period," from "child period" to "young period" and from "young period" to "adult period").”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Sakamoto wherein: the robot stores a setting of at least one of a personality parameter expressing a pseudo-personality or a growth parameter expressing pseudo-growth, and the gesture that the robot is to be caused to execute changes in accordance with at least one of the personality parameter or the growth parameter. Doing so provides a more realistic robot that changes over time with respect to its surroundings, as stated by Sakamoto (Col. 24 lines 16-22, where “The pet robot system 120 has the same configuration as the pet robot system 50 (FIG. 8) except that a pet robot 121 has a function of changing motions and actions as if the real animal "grew", in accordance with a history of operation inputs such as spurring and orders given with a sound commander from a user and histories of own actions and motions.”).
Regarding Claim 13, modified reference Oono teaches the imaging system according to claim 1, but is silent on wherein the robot includes: a housing in which a head is coupled to a torso by a coupler; and an exterior covering the torso.
However, Sakamoto teaches wherein the robot includes: a housing in which a head is coupled to a torso by a coupler (Col. 5 lines 14-19, where “The pet robot 2 is formed by coupling leg member units 11A through 11D with front right, front left, rear right, and rear left portions of a body member unit 10 and connecting a head member unit 12 and a tail member unit 13 to a front end and a rear end of the body member unit 10, as apparent from FIG. 1.”); and
an exterior covering the torso (Col. 6 lines 29-34, where “Accordingly, the robot system 1 is configured to allow the cover unit 3 to be fitted over the pet robot 2 in a fixed condition by fitting the cover unit 3 over the body member unit 10 of the pet robot 2 and tightening screws 4 into the tapped holes 10B of the body member unit 10 of the pet robot 10 through the screw holes 3B of the cover unit 3.”), (Note: The Examiner interprets the cover unit of Sakamoto as the exterior.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Sakamoto wherein the robot includes: a housing in which a head is coupled to a torso by a coupler; and an exterior covering the torso. Doing so allows for changeability and customization with the robot, improving user amusement and satisfaction, as stated by Sakamoto (Col. 1 lines 41-46, where “Furthermore, considering that a pet robot can wear a cover, not only its appearance can be changeable but also if it can perform different actions depending on the appearance, it is considered that such a pet robot will be capable of giving higher emotions of intimacy and satisfaction to users, which improve an amusement property in the pet robot.”).
14. Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oono et al. (US 20200306957 A1 hereinafter Oono) in view of Hayashi (US 20210283516 A1 hereinafter Hayashi), and further in view of Katzir et al. (US 20240414439 A1 hereinafter Katzir).
Regarding Claim 16, modified reference Oono teaches the imaging system according to claim 1, but is silent on wherein: each gesture of the plurality of predetermined gestures is associated with predetermined information indicating an execution time length of the gesture, and the processes further comprise, in response to two or more gestures being selected from among the plurality of predetermined gestures, causing the robot to execute the two or more gestures in a predetermined order, and controlling the camera to perform the video imaging of the robot for an imaging time length that is based on a sum of the execution time lengths of the two or more gestures so that the video imaging of the robot ends at a timing corresponding to a timing at which the robot ends execution of the two or more gestures.
However, Katzir teaches wherein: each gesture of the plurality of predetermined gestures is associated with predetermined information indicating an execution time length of the gesture ([0140] via “In some embodiments, a partial inspection plan (e.g. for a specific object) including, for example, a path (e.g. with time) of the imager, is calibrated using images acquired by the imager.”), ([0271] via “In some embodiments, outputs of a movement trajectory include, after the movement trajectory, one or more of: position, rotation angle, acceleration and duration of the movement trajectory. For example, position 870, rotation angle 822, acceleration 844 and duration 876 are outputs of first movement trajectory 842. For example, position 878, rotation angle 880, acceleration 882, and duration 884 are outputs of second movement trajectory 844.”), (Note: The Examiner interprets the durations 876, 884, etc. as the execution time lengths.), and
the processes further comprise, in response to two or more gestures being selected from among the plurality of predetermined gestures, causing the robot to execute the two or more gestures in a predetermined order ([0239] via “In FIG. 5A, imaging trajectories 544, 548, 550, 554, 556 are similar to and/or longer in time duration than translational trajectories 542,
546, 550. However, in an exemplary embodiment, duration of image acquisition is less than that of translational trajectories.”), ([0240] via “In some embodiments, at position A, imager
504 is within a home position e.g. within a dock 540. In some embodiments, between position A and position B, for example along trajectory 542, imager 502 is prepared for imaging. Between positions B and C (e.g. while moving along trajectory 544) imager 502 acquires one or more images of ROI 534. During preparing (e.g. between position A and position B), in some embodiments, the imager is accelerated or decelerated to a desired speed for collecting images.”), (Note: See Figure 5 of Katzir wherein each individual trajectory between positions A, B, C, etc. are in a predetermined order.), and
controlling the camera to perform the video imaging of the robot for an imaging time length that is based on a sum of the execution time lengths of the two or more gestures ([0264] via “In some embodiments, inspection plan 832 includes a plurality of movement trajectories. For example, a first movement trajectory 842, a second movement trajectory 844, up to an Nth movement trajectory 848. In some embodiments, a movement trajectory includes translational movement and/or rotational movement of the imager.”), ([0275] via “In an exemplary embodiment, one or more variables of inspection plan 832 is used to optimize a total duration of inspection plan 832. Which, in some embodiments, is equal to the sum of the durations of all of the imaging trajectories (e.g. duration 876, duration 844, and durations of imaging trajectories up to and including Nth movement trajectory 848).”) so that the video imaging of the robot ends at a timing corresponding to a timing at which the robot ends execution of the two or more gestures ([0264] via “In some embodiments, a movement trajectory includes translational movement and/or rotational movement of the imager.”), ([0275] via “In an exemplary embodiment, one or more variables of inspection plan 832 is used to optimize a total duration of inspection plan 832. Which, in some embodiments, is equal to the sum of the durations of all of the imaging trajectories (e.g. duration 876, duration 844, and durations of imaging trajectories up to and including Nth movement trajectory 848).”), (Note: The Examiner interprets that once the Nth movement trajectory 848 of Katzir is completed, that the inspection (i.e., imaging) is no longer is performed, as the Nth movement trajectory is the final movement trajectory.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Katzir wherein: each gesture of the plurality of predetermined gestures is associated with predetermined information indicating an execution time length of the gesture, and the processes further comprise, in response to two or more gestures being selected from among the plurality of predetermined gestures, causing the robot to execute the two or more gestures in a predetermined order, and controlling the camera to perform the video imaging of the robot for an imaging time length that is based on a sum of the execution time lengths of the two or more gestures so that the video imaging of the robot ends at a timing corresponding to a timing at which the robot ends execution of the two or more gestures. Doing so optimizes the imaging of the multiple gestures performed by inputting the known gestural parameters in order to generate an optimal imaging plan, as stated by Katzir ([0273] via “In some embodiments, inspection plan 832 is generated by optimizing one or more variables of the inspection plan. In some embodiments, the optimizing is for an object including individual object parameters. In some embodiments, the generating of the inspection plan includes as an input (or limit) the required image data of the object for inspection and a model of the object of inspection (e.g. including one or more features as illustrated and/or described regarding required image data 324, model 326 and inspection plan 332, FIG. 3B).”).
Examiner’s Note
15. The Examiner has cited particular paragraphs or columns and line numbers in the
references applied to the claims above for the convenience of the Applicant. Although the
specified citations are representative of the teachings of the art and are applied to specific
limitations within the individual claim, other passages and figures may apply as well. It is
respectfully requested of the Applicant in preparing responses, to fully consider the references
in their entirety as potentially teaching all or part of the claimed invention, as well as the
context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP
2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole,
including portions that would lead away from the claimed Invention. W.L. Gore & Associates,
Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851
(1984). See also MPEP §2123.
Conclusion
16. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
17. Any inquiry concerning this communication or earlier communications from the
examiner should be directed to BYRON X KASPER whose telephone number is (571)272-3895.
The examiner can normally be reached Monday - Friday 8 am - 5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is
encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Adam Mott can be reached on (571) 270-5376. The fax phone number for the
organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be
obtained from Patent Center. Unpublished application information in Patent Center is available
to registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for
more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BYRON XAVIER KASPER/Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights