DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6, 10-11, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Muck et al (US 20180181134, hereinafter Muck) in view of Manabe et al (US 20210156160, hereinafter Manabe).
Regarding Claim 1, Muck teaches:
a method for automating a rebar tying operation (see par. 0122 and Fig. 41) comprising:
a) introducing a ground-based robot into a concrete formwork including a rebar grid comprising a plurality of intersecting rebar elements configured in a grid formation (see at least " The assembly 10 and the software systems used to control its autonomous operation may be used in the following operations. In step one, the existing construction crew installs the screed rails 58 and lays out the bottom rebar mat 50 comprised of longitudinal and lateral rebar 52, 54. The crew manually ties about 10% of the intersections 56, including rebar overlap intersections. In step two, the assembly 10 is delivered to the job site and installed." in par. 0140 and Fig. 1) , wherein the ground-based robot comprises each of:
i) a chassis (see at least “tool actuator axis subassembly 300” in par. 0061)
ii) an intersection detection sensor attached to the chassis and configured to receive sensor data for detecting one or more rebar intersections on the rebar grid (see at least " A perception subsystem module 414 that receives data from stereo camera modules 406 communicates the data to intersection localization module 412. Intersection localization module 412 detects the occurrence of intersections and reports the information (i.e., there either is or is not a rebar intersection 56 at a particular location) to the grid map module 410. The primary goal of the perception subsystem in various aspects, is to identify and locate rebar intersections on the gird map." in par. 0114) ;
iii) a drive mechanism for transporting the ground-based robot (see at least “The gantry axis subassembly 100 contains a power source, such as motor 148, to power the gantry axis components, a secondary electronics box 160 for system control, and a feedback controlled drive system, including drive motor 184 to self-propel longitudinally along the rails 58 or ground in either continuous or step-&-settle motions.” in par. 0062);
iv) a rebar tying tool attached to the chassis and configured to tie the one or more rebar intersections (see at least “The tool end-effector 304 is carried on a frame 302 in the path of the tool actuator axis and self-aligns to the tool action site, which may, for example, be a rebar intersection 56. The end-effector 304 can be moved into position over the site of interest to act according to its intended function. In aspects wherein the tool end effector 304 is a rebar tying tool, the tool action would be to wrap and tie wire around the rebar intersection 56.” in par. 0066); and
v) a controller in communication with the intersection detection sensor, the drive mechanism, and the rebar tying tool; (see at least "The carrier subsystem computer 214 commands the carriage and actuator subsystem motion controllers to command remote starting of the power generator 256, and to perform all primary autonomous system functions such as perception, localization, planning, health and status messages, controller radio communication, perception sensor communication, and other system functions” in par. 0087)
b) identifying, by the intersection detector sensor, each of a plurality of rebar intersections in need of tying in the concrete formwork; (see at least “Intersection localization module 412 detects the occurrence of intersections and reports the information (i.e., there either is or is not a rebar intersection 56 at a particular location) to the grid map module 410. The primary goal of the perception subsystem in various aspects, is to identify and locate rebar intersections on the gird map.” in par. 0114 );
c) navigating, by the drive mechanism, the ground-based robot to each of the plurality of rebar intersections in need of tying in the concrete formwork; (see at least “the plan generator 418 receives data from the supervisor 400 regarding the grid map and determines where the subassembly motion axes go. The plan generator checks data from the grid map module, received through the position track module 408 through the supervisor 400 to locate, for example, the next intersection 56 and ask whether there is an obstacle there. If there is an intersection 56 and no obstacle, the plan generator module 419 will send a path plan to the coordinated motion control module 420 to go to the next intersection location on the grid map.” in par. 0124); and
d) tying, by the rebar tying tool, each of the plurality of rebar intersections in need of tying. (see at least “The supervisor 400 instructs the tool interface, for example, a tie gun or clip gun interface, to tie/clip or not to tie/clip.” in par. 0127 )
Muck does not appear to explicitly teach all of the following, but Manabe does teach:
i) a chassis adapted to be supported by the rebar grid; (see at least “self-propelled robot comprises at least wheel units each having a rear wheel and a front wheel that travel on any one rebar among a plurality of rebars laid out in a lattice shape, frame units mounted multiply and aligned on the right and left of the wheel units corresponding to array intervals of the rebar on which the wheel unit travel” in par. 0030 )
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck to incorporate the teachings of Manabe wherein the robot has a chassis and wheels that drive along the rebar grid. The motivation to incorporate the teachings of Manabe would be to bind rebars more stably and efficiently (see par. 0030)
Regarding Claim 2, Muck as modified by Manabe teaches:
the method of claim 1, wherein the intersection detection sensor is configured with some or all of:
a) one or more cameras;
b) lidar functionality; and/or
c) a range finder. (see at least “In various aspects, the perception sensor 340 may be any suitable three dimensional perception camera that utilizes stereo vision, laser scanning, laser time-of-flight, or any other mean of imaging a scene in three dimensions. The perception sensor may include, for example, a pair of stereo vision cameras 342.” in par. 0091)
Regarding Claim 3, Muck as modified by Manabe teaches:
the method of claim 2, wherein:
Muck further teaches: a) the intersection detection sensor is configured with one or more cameras (see at least “In various aspects, the perception sensor 340 may be any suitable three dimensional perception camera that utilizes stereo vision, laser scanning, laser time-of-flight, or any other mean of imaging a scene in three dimensions. The perception sensor may include, for example, a pair of stereo vision cameras 342.” in par. 0091); and
b) a position and orientation of the ground-based robot in the concrete formwork is determined by analysis of camera positions derived from the one or more cameras (see at least "The images are communicated from the cameras to the computer 214 to detect the mat, identify the relevant grid, and localize the rebar intersections 56. The relevant grid may be identified in three steps, by segmenting the foreground, detecting the grid orientation, and detecting the intersections. The perception system also perceives obstacles, if any, that might be on the rebar mat by detecting the obstacles above or below the grid and localizing the obstacles." in par. 0108 and “The navigation infrastructure includes a position tracking module 408 that is responsible for maintaining the current transform network and reporting where components of the assembly 10 are at any given time based on feedback from the various positioning sources. The position tracker module 408 receives data from various positioning sources, such as the gantry idler wheels 152 and carrier idler wheels 254, and from a rebar grid map module 410. Feedback from the idler wheels 152 provide data for correction of the drive wheel 150 positions to match the absolute position provided by the idler wheel feedback.” In par. 0113 and “In use, the stereo cameras 342 capture the image within the fields of view 290 and 292 below each camera 342 at the same time and each sends the image data through a video processor which identifies features in the image, determines the distance between the two images and how far the cameras and the features in the images have moved since the prior images. The software triangulates the data to provide a three dimensional (3-D) view of the target site and its surroundings.” In par. 0115 and “The tool position tracking functions, in various aspects, may include automatic slip corrections based on the idler encoder feedback, for example, through module 404. GPS sensor feedback may in various aspects, be integrated to maintain correlation between the grid map positions before and after software restart actions (i.e. shift changes). All of these sensor inputs will be collected together in the grid map via a position tracking filter module 408.” In par. 0136).
Regarding Claim 4, Muck as modified by Manabe teaches:
the method of claim 1,
wherein when the ground-based robot navigates around the concrete formwork data is generated for each of:
a) position of rebar elements laid;
b) spacing of rebar elements laid;
c) tied intersections; and
d) intersections remaining to be tied. (see at least " The modeling function maintains a model of the rebar map by saving the perceived intersection and obstacle location data and, as the assembly moves and more images are processed, constructs a rebar, or work site, grid map, including intersection 56 localizations. The locations of detected obstacles are also saved on the developing grid map." in par. 0109 and “The grid map is a dynamic data structure that adds and stores information to the grid map as image data is received and analyzed. The grid map maintains a list of the location of all of the tied and untied intersections 56 within the field of view of the stereo cameras 342 communicated via module 406 to the perception subsystem 414.” In par. 0122)
Regarding Claim 6, Muck as modified by Manabe teaches:
the method of claim 1, further comprising:
Muck further teaches: a) generating data from the rebar tying operation; and b) transferring the data to one or more project stakeholders for use in documentation associated with the concrete formwork and rebar tying operation. (see at least " If an error is detected in this process the error is reported to the operator." in par. 0143)
Regarding Claim 10, Muck as modified by Manabe teaches:
the method of claim 1,
Muck further teaches: wherein the ground-based robot is further configured with a servomotor configured for control of each of (see at least " In various aspects, there may be one drive power supply 240 for each axis of motion. In the embodiment of the assembly shown, there are four drive power supplies 240, one for each drive motor 184 on each side of the truss 102, one for the carrier subassembly drive motor 226, and one for the tool actuator drive motor 315. " in par. 0086) :
a) angular position; (see at least " Because the first longitudinal path along which the gantry travels will not always be straight, the second path, transverse to the first path will not always be at a 90° angle relative to the first path. The first path may curve, as shown in FIG. 2, and the second path will extend at an angle, Θ, from the first path." in par. 0063)
b) linear position (see at least "The carriage 202 carries the tool actuation subassembly 300. The tool actuation subassembly 300 includes a motion actuator, which may be in the form of a linear motion actuator, a delta actuator, or a parallel kinematic actuator. " in par. 0095) ;
c) velocity (see at least "In addition, the software may include features for improving the system productivity by controlling the motion control step & settle speed, increasing the speed in real time in response to work surface conditions where appropriate" in par. 0111 ) ; and
Muck does not appear to explicitly teach all of the following, but Manabe does teach:
d) acceleration. (see at least " As illustrated in FIG. 4, the control unit 400 includes a wheel drive control portion 420 therein. The wheel drive control portion 420 also includes a travel control portion 421 configured to control travel motions related to acceleration/deceleration and to stoppage of a wheel driving portion 510 of the driving unit 500 described later and a forward/retreat switching control portion 422 configured to control and switch a forward motion with a retreat motion in the wheel driving portion 510 of the driving unit 500 described later.." in par. 0046)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck to incorporate the teachings of Manabe wherein the robot has motors that control the acceleration of the end effector. The motivation to incorporate the teachings of Manabe would be to bind rebars more accurately and steadily (see par. 0020)
Regarding Claim 11, Muck as modified by Manabe teaches:
the method of claim 1,
Muck further teaches: wherein the ground-based robot is configured with a tracker for positioning the rebar tying tool at each identified rebar intersection. (see at least "A global positioning system (GPS) unit may be employed to adjust for the gantry truss location relative to the rebar." in par. 0135 and “GPS sensor feedback may in various aspects, be integrated to maintain correlation between the grid map positions before and after software restart actions (i.e. shift changes). All of these sensor inputs will be collected together in the grid map via a position tracking filter module 408.” In par. 0136)
Regarding Claim 13, Muck as modified by Manabe teaches:
the method of claim 1,
Muck further teaches: wherein identification of the rebar intersection comprises user input. (see at least "The assembly operator performs an initialization of the assembly 10 to the mat 50 and then initiates continuous or intermittent tying operations. The operator initializes the assembly 10 by first positioning the gantry and carrier axes, via remote control, at the start of the mat 50. The operator then manually verifies or adjusts the tool actuator axis stroke to fully engage the mat 50 with appropriate force, setting the desired stroke." in par. 0141)
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Muck et al (US 20180181134, hereinafter Muck) in view of Manabe et al (US 20210156160, hereinafter Manabe) and Shah et al (US 20190311203; hereinafter referred to as Shah)
Regarding Claim 5, Muck as modified by Manabe teaches:
the method of claim 1, further comprising:
Muck and Manabe do not appear to explicitly teach all of the following, but Shah does teach:
a) generating data from the operation (see at least " The training process may include receiving the training images. For example, FIG. 9 shows an embodiment of an aerial monitoring method 900 in which a step 902 includes receiving training images of training objects with predefined object features. The image processing engine records at least a portion of these training images with an assigned category for each training image in a database for future retrieval during image processing." in par. 0044) ; and
b) using the data to train the robot for identification in a subsequent operation. (see at least " The image processing engine records at least a portion of these training images with an assigned category for each training image in a database for future retrieval during image processing." in par. 0044 and “The image processing engine uses the training images to generate a CNN for identifying object features in monitoring images. For example, aerial monitoring method 900 includes a step 904 of building a convolutional neural network using the training images. Then, the image processing engine practices using the CNN with training images. The image processing engine uses the feedback from the practice to improve the application of the model.” In par. 0046)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck as modified by Manabe to incorporate the teachings of Shah wherein the robot records data during operation for future training improvements to the object identification model, in order to arrive at the same control of the rebar tying robot system taught by Muck. The motivation to incorporate the teachings Shah would be to improve the probability of correctly identifying object features in images (see par. 0055)
Claim(s) 7-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Muck et al (US 20180181134, hereinafter Muck) in view of Manabe et al (US 20210156160, hereinafter Manabe) and Hetrick et al (US 10597264, hereinafter Hetrick)
Regarding Claim 7, Muck as modified by Manabe teaches:
the method of claim 1,
Muck and Manabe do not appear to explicitly teach all of the following, but Hetrick does teach:
wherein a marking is generated for the concrete formwork and for rebar elements inside the formwork, and wherein the ground-based robot navigates to a rebar intersection by visual recognition of the marking. (see at least " Determination of the designated location may be done in advance in accordance with a pre-planned construction plan, or may be a location determined relative to the location of existing rebar, or relative to two or more marked points" in col. 8 lines 39-43 and “The sensing function may, for example, use the cameras 342 or other image data sources to find the fiducial markers 70 positioned at the work site 12 and on portions of the apparatus components, such as the bottom 322 of the sliding rail 306, and use the data for surface estimation, to locate and verify rebar positions, to detect obstacles and for perception diagnostics.” In col 23 lines 61-67 )
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck as modified by Manabe to incorporate the teachings of Hetrick wherein markers are spaced throughout the worksite and detected by the robot cameras to help it navigate to rebar intersections. The motivation to incorporate the teachings of Hetrick would be to improve the accuracy of the robot model of the surroundings (see col 26 line 55-63).
Regarding Claim 8, Muck as modified by Manabe and Hetrick teaches:
The method of claim 7,
Muck and Manabe do not appear to explicitly teach all of the following, but Hetrick does teach:
wherein the marking is generated by a human. (see at least " The control system's sensing function may receive sensory signals from image data sources to find markers pre-positioned at the work site and on portions of one or more of the gantry, tram, actuation and gripper subassemblies to generate the sensing data." in col. 4 lines 41-45 )
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck as modified by Manabe to incorporate the teachings of Hetrick wherein markers pre-positioned by operators throughout the worksite and detected by the robot cameras to help it navigate to rebar intersections. The motivation to incorporate the teachings of Hetrick would be to improve the accuracy of the robot model of the surroundings (see col 26 line 55-63).
Regarding Claim 9, Muck as modified by Manabe and Hetrick teaches:
the method of claim 7,
Muck further teaches: wherein the marking is generated by the ground-based robot. (see at least "The perception system may also include the ability to sense color. The addition of color sensing enables the software objective to identify already tied intersections 56 on rebar mat 50 (for example, yellow wire on green or blue epoxied rebar 52, 54)." in par. 0094 )
Claim(s) 12, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Muck et al (US 20180181134, hereinafter Muck) in view of Manabe et al (US 20210156160, hereinafter Manabe) and Fan et al (US 20200101616; hereinafter referred to as Fan)
Regarding Claim 12, Muck as modified by Manabe teaches:
the method of claim 1,
Muck and Manabe do not appear to explicitly teach all of the following, but Fan does teach:
wherein the ground-based robot is configured for moving and placing a rebar element in the concrete formwork. (see at least “Then the micro-manipulator with an auto rebar tier to position joints precisely. In one embodiment, one or more piezoelectric motor or voice coil motor.” in par. 0043).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck as modified by Manabe to incorporate the teachings of Fan wherein the automated rebar tying robot precisely positions the joints before tying. The motivation to incorporate the teachings of Fan would improve location precision of rebar joints (see par. 0043).
Regarding Claim 14, Muck as modified by Manabe teaches:
the method of claim 1,
Muck and Manabe do not appear to explicitly teach all of the following, but Fan does teach:
wherein the ground-based robot is configured to determine a size of a rebar element. (see at least “The typical size of the rail grid is around 4.9 m×3 m and the typical diameter of rebar 10-12 mm. In one embodiment of the present invention, there is provided a micro-macro manipulator with AI vision. The macro manipulator comprises large scale 2-axis gantry rails. The vision sensors or cameras are adapted to recognize the rebar joint or intersection position, e.g. depth camera 30 cm above the workpiece grid can sense bar-shape objects with a diameter of 10 mm.” in par. 0043)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck as modified by Manabe to incorporate the teachings of Fan wherein the automated rebar tying robot uses the size of the rebar along with the position of the rebar to automatically identify and tie untied rebar joints. The motivation to incorporate the teachings of Fan would be to save time and cost by automating the process of rebar joint identification and tying (see par. 0002-0003).
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Muck et al (US 20180181134, hereinafter Muck) in view of Manabe et al (US 20210156160, hereinafter Manabe) and Newman et al (US 20200198122, hereinafter Newman)
Regarding Claim 15, Muck as modified by Manabe teaches:
the method of claim 1, wherein:
Muck and Manabe do not appear to explicitly teach all of the following, but Newman does teach:
a) the ground-based robot is in communication with a base station and b) the base station is configured to manage operation of a plurality of ground-based robots in the concrete formwork simultaneously. (see at least “Also shown is a watch-tower 2839 configured to monitor the various robots 2801 and redirect activities as needed. The watch-tower 2839 may include a supervisory computer and/or a human manager depending on the type of monitoring needed. The watch-tower 2839 includes a localization system 2819 so that the supervisor can transmit a hailing message when needed, thereby causing the robots 2801 to report their location using localization signals and their identity using wireless messages, on request.” in par. 0132 and “cooperative robots may also be employed in direct construction tasks such as rebar welding” in par. 0134).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot taught by Muck as modified by Manabe to incorporate the teachings of Newman wherein a group of construction robots are supervised via a watch tower, in order to arrive at performing the same control with the robots taught by Muck and Manabe. The motivation to incorporate the teachings of Newman would be to improve coordination and localization of multiple robots (see par. 0132, 0134).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DYLAN M KATZ whose telephone number is (571)272-2776. The examiner can normally be reached Mon-Thurs. 8:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DYLAN M KATZ/Examiner, Art Unit 3657