CTNF 19/079,790 CTNF 83818 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claim (s) 1-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kikkeri et al., US PG Pub 2016/0354927 A1., (hereafter Kikkeri) in view of Doettling et al., US PG Pub 2015/0049911 A1., (hereafter Doettling) . Regarding claim 1 where it is disclosed by Kikkeri to have a robotic manipulator which is controlled by the use of cameras to prevent the collision of the manipulator arm in the presence of another moving object, as shown in at least figures 1 and 6. Where this is read upon by applicants claim to, “ A robot system [at least figure 1 shows a robotic arm] , comprising: a robot [at least figure 1 object labeled “102” described in at least paragraph 0026] ; and a vision device including an imaging module capturing a real-time video image of a work site of the robot [at least paragraph 0026 where it states the use of “depth sensing camera system 104” which is shown in figure 1] , a partition module dividing the captured video image into a plurality of different areas [at least figure 3 box “310” where the picture frames are captured and are interpreted as the system dividing the image area in to different zones to capture moving object as it moves, see at least paragraphs 0005-0006] , a recognition module recognizing whether a person has entered the plurality of different areas [see at least paragraph 0005-0006 where the person is sensed as they move through the cameras capture area] , and a control module communicating with the robot and controlling the robot based on a recognition result of the recognition module [see at least figures 3 & 6 where the robot is connected to controller that recognize moving objects via the use of camera] , the plurality of different areas includes a first virtual barrier area surrounding the robot [at least paragraphs 0026 & 0030-0031 as also shown in figure 1 aeras labeled 108 and 110] , when the recognition module recognizes that a person has entered the first virtual barrier area the control module sends a stop operation control instruction to the robot, stopping the robot immediately [at least figure 6 and paragraphs 0017, 0026, 0030-0031, 0034 & 0037].” Where it is disclosed by Kikkeri to have their system detect moving objects it is not specifically disclosed by Kikkeri to have their system detect a “person”. Doettling is directed to the method and system for safeguarding a hazardous working area of an automated machine which using a camera system can detect a person and stop the robotic arm as shown in at least figures 1-3. Doettling does teach their system being able to detect a person in the zones labeled 24 & 30 in figure 2 and described in at least paragraphs 48-49, 51-52 & 54. The system of Doettling described the classification of a person in the area in at least figure 3 also described in at least paragraphs, 11-12 & 57-61. Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 2 where it is disclosed by both Kikkeri in at least paragraphs 17, 30 & 37 as well as Doettling in at least paragraph 5 to have their system also having, “the plurality of different areas further include a second virtual barrier area surrounding the first virtual barrier area, when the recognition module recognizes that the person has not entered the first virtual barrier area but recognizes that the person has entered the second virtual barrier area, the control module sends a deceleration operation control instruction to the robot to reduce an operation speed of the robot.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 3 where it is disclosed by both Kikkeri and Doettling to have their system include the feature of, “the plurality of different areas also includes a safety area located outside the second virtual barrier area, when the recognition module recognizes that the person has not entered the first virtual barrier area and the second virtual barrier area but recognizes that the person has entered the safety area, the control module sends a normal operation control instruction to the robot allowing the robot to operate normally.” Where this is disclosed by Kikkeri in at least paragraphs 26, 30-32 & 54 where they describe a “warning zone” and by Doettling in at least paragraph 51-52. Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 4 where it is disclosed by Kikkeri in at least paragraphs 26, 30-32 & 35 to have, “production equipment located outside the first virtual barrier area but in the second virtual barrier area, the robot located in the first virtual barrier area is applied to the production equipment.” Regarding claim 5 where it is disclosed by Kikkeri in at least paragraphs 17, 29, 30-31 and 40 to have, “the robot is a mobile robot that docks to the production equipment and other different production equipment.” Regarding claim 6 where all the limitations of claim 2 are disclosed by Kikkeri and Doettling as described above. Where it is further disclosed by Doettling in at least figure 3 and the associated paragraphs to have their system also have, “the recognition module only recognizes features of a human body and cannot recognize features of objects other than the human body, the recognition module allows objects other than features of the human body to enter the first virtual barrier area and the second virtual barrier area.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 7 where all the limitations of claim 6 are disclosed by Kikkeri and Doettling as described above. Where it is further disclosed by Doettling in at least figure 3 as well as paragraphs 27 & 67 to have their system also have, “the features of the human body include head features, body features, hand features, and foot features.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 8 where all the limitations of claim 7 are disclosed by Kikkeri and Doettling as described above. Where it is further disclosed by Doettling in at least figure 3 as well as paragraphs 27 & 67 to have their system also have, “when any one of the features of the human body are recognized in the first virtual barrier area or the second virtual barrier area, the recognition result of the recognition module is that the person has entered the first virtual barrier area or the second virtual barrier area.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 9 where all the limitations of claim 7 are disclosed by Kikkeri and Doettling as described above. Where it is further disclosed by Doettling in at least figure 3 as well as paragraphs 27 & 67 to have their system also have, “the imaging module includes a camera having hardware, the partition module, the recognition module, and the control module are functional modules integrated into the hardware, the control module is connected to the robot through a signal line to communicate with the robot.” Where in Doettling if the classifier as shown in figure 3 does not recognize a human or its associated body parts then the system would not stop and continue to have the robot move. Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 10 where it is both disclosed by Kikkeri in at least figures 3-6 and Doettling in at least figures 1-3 to have their system include the components associated with, “the robot includes moving parts, the moving parts include a robotic arm, when the recognition module recognizes that the person has entered the first virtual barrier area, the control module controls the moving parts to immediately stop the moving parts from moving.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 11 where it is both disclosed by Kikkeri in at least figures 1-6 and Doettling in at least figures 1-3 to have their system include the components associated with, “the robot includes moving parts, the moving parts include a robotic arm, when the recognition module recognizes that the person has entered the first virtual barrier area, the control module controls the moving parts to immediately stop the moving parts from moving.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 12 where it is both disclosed by Kikkeri in at least figures 1-6 and Doettling in at least figures 1-3 to have their system include the components associated with, “the robot includes moving parts, the moving parts include a robotic arm, when the recognition module recognizes that the person has not entered the first virtual barrier area but recognizes that the person has entered the second virtual barrier area, the control module controls the moving parts to reduce a movement speed of the moving parts.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 13 where all the limitations of claim 2 are disclosed by Kikkeri and Doettling as described above. Where it is further disclosed by Doettling in at least figure 3 as well as paragraphs 27 & 67 to have their system also have, “the robot includes moving parts, the moving parts include a robotic arm, when the recognition module recognizes that the person has not entered the first virtual barrier area and the second virtual barrier area, the moving parts of the robot continue moving normally.” Where in Doettling if the classifier as shown in figure 3 does not recognize a human or its associated body parts then the system would not stop and continue to have the robot move. Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of 12success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 14 where it is further disclosed by both Kikkeri in at least paragraph 0039 and figure 3 as well as Doettling in at least paragraphs 0045-0047 and figure 1 to have, “the robot system comprises a plurality of vision devices, the plurality of vision devices are respectively set at multiple different locations and in multiple different orientations.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of 12success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 15 where it is further disclosed by both Kikkeri in at least figure 6 as well as Doettling in at least figure 3 to have, “when the robot receives the stop operation control instruction from any of the plurality of vision devices, the robot immediately stops working.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of 12success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 16 where it is further disclosed by both Kikkeri in at least figure 6 as well as Doettling in at least figure 3 to have, “when the robot does not receive the stop operation control instruction but receives a deceleration operation control instruction from any of the plurality of vision devices, the robot immediately reduces an operation speed.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of 12success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling. Regarding claim 17 where it is further disclosed by both Kikkeri in at least figure 6 as well as Doettling in at least figure 3 to have, “when the robot receives both the stop operation control instruction and the deceleration operation control instruction, the robot immediately stops working.” Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri by the teachings of Doettling, where they are both directed to the same field of endeavor of helping prevent collisions between robot manipulators and other object whilst they are moving in a work space. Where one with a reasonable expectation of 12success would have been motivated to modify Kikkeri by the use of a known technique to improve similar devices in the same way, as taught by Doettling. Where in this instance the modification of Kikkeri to specifically detect and classify humans that move into the working space of an active robot to prevent the collision between the robot arm and the human as to prevent serious injury to the human when the robot might collide with them, see at least paragraph 2 of Doettling . 07-22-aia AIA Claim (s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kikkeri and Doettling as applied to claim 1 above, and further in view of Meinherz et al., US PG Pub 2012/0095575 A1., (hereafter Meinherz) . Regarding claim 18 where all the limitations of claim 1 are disclosed by Kikkeri and Doettling, as described above. However, neither Kikkeri and Doettling specifically disclose the further limitation of having their system include the feature of, “the recognition module is an artificial intelligence vision system that has undergone large-scale model training and can effectively recognize human features and behaviors.” Meinherz discloses a system that can detect a humans movement by using artificial intelligence as disclosed in at least paragraphs 0035, 0045 & 0046 to have, “the recognition module is an artificial intelligence vision system that has undergone large-scale model training and can effectively recognize human features and behaviors.” Thus it would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify Kikkeri and Doettling by the teachings of Meinherz where they are all directed the same field of endeavor of controlling robotic movement. Where one with a reasonable expectation of success would have looked to modify Kikkeri and Doettling by the use of a known technique to improve similar devices in the same way, as taught by Meinherz. Where in this instance the modification of Kikkeri and Doettling whom do not specifically use artificial intelligence (AI) for teaching the robotic system to identify humans would allow the improvement od detecting humans and become quicker at responding to said detection by the use of the AI, as taught by Meinherz . Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claims 9 & 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding claims 9 & 13 where applicant claims, “when none of the features of the human body are recognized in the first virtual barrier area or the second virtual barrier area, the recognition result of the recognition module is that the person has not entered the first virtual barrier area or the second virtual barrier area[.]” and “the robot includes moving parts, the moving parts include a robotic arm, when the recognition module recognizes that the person has not entered the first virtual barrier area and the second virtual barrier area, the moving parts of the robot continue moving normally[.]”, respectively. The currently presented claims have been constructed as a negative limitation, where this is permissible as per the MPEP, it is however not permissible to construct the negative limitation as so the scope of the claim can not be ascertained. In this instance the claims merely state what is not being detected but do not provide any limitation or scope as to what is being detected and hence the claims are ambiguous. The metes and bounds of the claims can not be ascertained. However for the purpose of compact prosecution the claims have been examined as best understood, see above rejection. It is suggested to applicant to amend the claims to provide some meaningful limitations to the claim. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to BHAVESH V AMIN whose telephone number is (571)270-3255. The examiner can normally be reached M-Thur, 8-6:30, EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached at (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. BHAVESH V. AMIN Primary Examiner Art Unit 3657 /BHAVESH V AMIN/Primary Examiner, Art Unit 3657 Application/Control Number: 19/079,790 Page 2 Art Unit: 3657 Application/Control Number: 19/079,790 Page 3 Art Unit: 3657 Application/Control Number: 19/079,790 Page 4 Art Unit: 3657 Application/Control Number: 19/079,790 Page 5 Art Unit: 3657 Application/Control Number: 19/079,790 Page 6 Art Unit: 3657 Application/Control Number: 19/079,790 Page 7 Art Unit: 3657 Application/Control Number: 19/079,790 Page 8 Art Unit: 3657 Application/Control Number: 19/079,790 Page 9 Art Unit: 3657 Application/Control Number: 19/079,790 Page 10 Art Unit: 3657 Application/Control Number: 19/079,790 Page 11 Art Unit: 3657 Application/Control Number: 19/079,790 Page 12 Art Unit: 3657 Application/Control Number: 19/079,790 Page 13 Art Unit: 3657 Application/Control Number: 19/079,790 Page 14 Art Unit: 3657 Application/Control Number: 19/079,790 Page 16 Art Unit: 3657 Application/Control Number: 19/079,790 Page 17 Art Unit: 3657 Application/Control Number: 19/079,790 Page 18 Art Unit: 3657
Prosecution Insights