Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Priority
Acknowledgment is made of applicant's claim for domestic benefit based on provisional application 63/404,313 filed on September 7, 2022.
Content of Specification
The specification is objected to for the following informalities: Specification paragraph 0007 states a “machine language (ML) module,” but should read “machine learning (ML) module.”
Claim Objections
Claim 2 is objected to because of the following informalities: Claim 2 contains the element: “a machine language (ML) module,” but should read “machine learning (ML) module.” Appropriate correction is required.
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 1, 9, 10, and 11 invoke 35 U.S.C. 112(f) because they use generic placeholders “configuration module” and a “machine learning (ML) module” coupled with functional language “configured to implement the adjusted nest offset or part routing” and “configured to analyze the operational data” respectively that is not modified by sufficient structure, material, or acts for performing the claimed function. The written description of the specification implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function. A review of the specification shows that the following implies that the corresponding structure described in the specification for the 35 U.S.C. 112(f) limitation "of a configuration module." Paragraph 0042 states that: “The display/interface 215 provides output information to the end user 240. The processor 220 processes data from the PLC 205 and provides processing power to the configuration module 210 for performing embodiments of the method of managing automation systems described herein. The processor 220 also provides output to the display.” Paragraph 0049 states: “a machine learning (ML)module/processor.” Thus the configuration module is a hardware device accepting power, and the machine learning module may be a processor which inherently has structure.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may:
(1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or
(2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 4 is are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention. Claim 4 recites the limitation "determining if an ML result is available based on a current ML model.” However, the claim or the specification does not defined or narrow what it means for a machine learning result to be available. The element “machine learning (ML) result is available based on a current ML model" is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree its meaning, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention for the term. The most indefinite term for the element is: “is available” based on a current ML model " renders the claim indefinite because it is unclear how a machine learning model becomes available. The specification does not provide a standard for ascertaining the requisite degree the meaning of the term. Appropriate action is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 3, 5, 6, 8 – 10, 12, 13, and 15 are rejected under 35 U.S.C. 102(a)(1) or 102(a)(2) as being anticipated by Matl et al. (PG Pub. No. 20220048707), herein “Matl.”
Regarding claim 1,
Matl teaches a method for dynamic nest and routing adjustment in an automation system, the method comprising: (Abstract: “A system and method for a dynamic robot kitting line that can include: processing a set of order requests and setting a packing fulfillment plan process for a robotic kitting line, wherein robotic kitting line comprises a conveyor system…” Par. 0002.)
operating the automation system; (Par. 0023: “In one variation, the system and method may additionally or alternatively involve the design and operation of an automated robotic kitting system, such as described herein, that is customized for streamlined fulfilling packing orders that can include various quantities of a variety of item types.”)
receiving operational data related to the automation system from a conveyor system and at least one automation station; (Par. 0102: “The packing control system can additionally be in communication with a sensing system which may be monitoring item bins, tote position, item placement in totes, item position in item bins, and/or any suitable aspect. The system may include a sensing system which can include one or more types of sensors such as camera/imaging devices, proximity sensors, contact sensors, and/or other suitable types of sensors.” Par. 0114: “The digital orders will generally convey what items are to be collected together for an order. The digital orders may additionally specify timing or priority properties for a particular order. The digital orders may additionally have item arrangement details, and/or other constraints or details related to packing of items for an order.” Par. 0115: “Configuration of the packing control system for processing the set of order requests and setting the packing fulfillment plan process for the robotic kitting line may include a variety of computer-implemented processes that determine properties of the packing fulfillment plan. The set of computer-implemented processes used and the objectives of those processes may use application dependent optimization processes.” See also Par. 0092.
analyzing the operational data to determine if adjustment of nest offsets or routing of parts is required; determining an adjusted nest offset or part routing based on the operational data; (Par. 0116: “In one example, a kitting usage scenario may have the configuration of the packing control system 130 evaluate conditions of the robotic kitting line and the order requests to minimize swaps of item bins and to enhance throughput in processing the kits. The packing control system 130 may implement one or more analysis processes to output order-to-tote assignments and/or item bin positioning to enhance balance of work performed by the robotic workcells (e.g., reducing instances when one robotic workcell is holding up work by other robotic workcells) and to reduce changes to item bin positioning (e.g., reducing time and labor involved in updating item bins).” Par. 0187: “Managing operation of the robotic kitting line may additionally include various processes related to the sensing the state of the robotic kitting line such as monitoring item bin status and monitoring item placement success (e.g., detecting grasping errors). Sensing the state of the robotic kitting line may be performed through various sensing approaches. In one preferred approach, sensing state includes collecting image data and processing the image data to determine status information. This may include performing computer vision on visual image data. This may alternatively include collecting a depth image (or other 3D or other forms of depth information) and performing some analysis on this multi-dimensional image information. For example, the quantity of items in an item bin may be estimated based on analysis of the depth map of the contents in the item bin.” See also Par. 0026, 0103, 0164, and 0174. Examiner Note – Specification paragraph 0010 and 0011 define nest offset as by controlling an accessory on a pallet of the conveyor system or by controlling a moving element of the conveyor system.)
implementing the adjusted nest offset or part routing via a configuration module of the automation system; and return to operating the automation system. (Par. 0096: “In one variation, the conveyor system 120 may include a tote hold system, which functions to divert a tote to a holding station as shown in FIG. 5. A robotic workcell 110 may each have a tote hold system with a plurality of holding stations. As with the item bins and the totes, the totes in the holding station are preferably within a reachable area of the robotic system 111. A tote diverter mechanism could be a directionally controlled conveyor system, a piston, or other mechanism to push or redirect an item tote 121 elsewhere or any suitable mechanism to move a tote between a position on a conveyor system and a holding station. In some cases, the tote hold system may be used to temporarily hold a tote. The tote hold system could also be used to reorganize or adjust the order of totes. Alternatively, the robotic system 111 may be configured to selectively manipulate an item tote 121 and divert the tote between the conveyor system 120 and a hold station.” See also Par. 0129, 0131, 0165, 0170, and 0171.)
Regarding claim 2,
The previously cited reference(s) teach the limitations of claim 1 which claim 2 depends. Matl also teaches that wherein the analyzing and determining are performed by a machine language (ML) module. (Par. 0026: “Related to the variety of items, the planned fulfillment of the orders implemented through the system and method can be based at least in part on the predictive data modeling for robotic pick-and-place handling. For example, planned packing of items can differ depending on if the machine learning models indicate higher confidence or lower confidence in handling the item. For example, grasp planning data modeling (based on computer vision-based analysis of items) may have higher confidence for items with packaging that visually are classified to be easier to grasp compared to an item with visual characteristics where grasping is challenging or where it is unfamiliar. The state of AI or machine learning models as it relates to items in the orders can alter how the system and method plan out distributing items across the various bins.” See also paragraphs 0033, 0106, 0143, 0166, 0172, and 0200.)
Regarding claim 3,
The previously cited reference(s) teach the limitations of claim 2 which claim 3 depends. Matl also teaches that further comprising providing feedback to update the ML module based on operational data received after the implementing. (Par. 0143: “The method is preferably implemented in combination with a robotic kitting system such as the one described above but may alternatively be used in combination with any suitable robotic kitting system that includes a plurality of robotic picker systems arranged along the length of a conveyor system, and where each robotic picker system has a plurality of item bins within grasping range. Preferably, there is a redundancy of at least a subset of item bins for one type of item. The method can adapt to different and new packing order objectives. The method may additionally or alternatively adapt operation of the robotic kitting line to the real-time conditions such as: updates to training of machine learning models used in grasp planning of a robotic system, success rates of packing a type of item, configuration of a robot (e.g., type of end effector), and/or other conditions.”)
Regarding claim 5,
The previously cited reference(s) teach the limitations of claim 1 which claim 5 depends. Matl also teaches that the automation system includes a plurality of automation stations and the receiving operational data comprises receiving data relating to the failure of at least one of the plurality of automation stations, indicating a need for an adjustment in routing of parts. (Par. 0112: “Another potential packing fulfillment objective could be to reduce dependence on post-processing correction. This objective may optimize for higher probabilities of the system successfully automatically resolving packing mistakes. For example, this objective could result in more redundant item bins being placed downstream so that placement errors can be automatically corrected if an upstream robotic workcell 100 fails to properly place an item.” See paragraphs 0125, 0129 – 0131, 0164, 0165, 0171, 0187, 0193 – 0195.)
Regarding claim 6,
The previously cited reference(s) teach the limitations of claim 1 which claim 6 depends. Matl also teaches that the receiving operational data comprises receiving vision data of the at least one automation station, indicating a need for an adjustment of nest offsets. (Par. 0026: “Related to the variety of items, the planned fulfillment of the orders implemented through the system and method can be based at least in part on the predictive data modeling for robotic pick-and-place handling. For example, planned packing of items can differ depending on if the machine learning models indicate higher confidence or lower confidence in handling the item. For example, grasp planning data modeling (based on computer vision-based analysis of items) may have higher confidence for items with packaging that visually are classified to be easier to grasp compared to an item with visual characteristics where grasping is challenging or where it is unfamiliar. The state of AI or machine learning models as it relates to items in the orders can alter how the system and method plan out distributing items across the various bins.” Par. 0132: “The sensing system functions to collect data of the objects and the environment. The sensing system preferably includes an imaging system, which functions to collect image data. The imaging system preferably includes at least one imaging device with a field of view of a region of interest within the robotic kitting line such as the item bins and/or the totes on the conveyor system 120. The imaging system may additionally include multiple imaging devices used to collect image data from multiple perspectives of a distinct region, overlapping regions, and/or distinct non-overlapping regions. The set of imaging devices (e.g., one imaging device or a plurality of imaging devices) may include a visual imaging device (e.g., a camera). The set of imaging devices may additionally or alternatively include other types of imaging devices such as a depth camera. Other suitable types of imaging devices may additionally or alternatively be used.” Par. 0187.)
Regarding claim 8,
The previously cited reference(s) teach the limitations of claim 1 which claim 8 depends. Matl also teaches that the adjustment of nest offset is performed by controlling a moving element of the conveyor system. (Par. 0101: “The packing control system 130 as discussed can be communicatively coupled to the robotic system in of each robotic workcell 110 and the conveyor system 120. In one variation, the packing control system 130 can be fully control in control of both the robotic workcell no and the conveyor system 120. In another variation, the packing control system 130 can be in control of the robotic workcell no and an observer of a conveyor system that is controlled external to the system. For example, the system may operate around a conveyor line that is continuously operated, pre-configured to move in a certain way, or controlled by another control system. The packing control system 130 may alternatively be interested with the robotic workcells 110 and/or the conveyor system 120 in any suitable configuration.” Par. 0123: “Configuration to direct loading of the set of item bins may include configuration to direct an automated item delivery robot to transport an item bin to a specified location of a workcell within the set of workcells 110. This can include communicating to an item delivery robot. This may alternatively include actively controlling an item delivery robot.” Par. 0038, 0084, 0088, 0098, 0099, and 0101.)
Regarding claims 9 and 10, they are directed to a system or apparatuses to implement the method of steps set forth in claims 1 - 3. Matl teaches the claimed method of steps in claims 1 – 3. Claim 9 teaches a machine learning module which is embodied in claim 2. Therefore, Matl teaches the system or apparatuses to implement the claimed method of steps in claims 9 and 10.
Regarding claims 12, 13, and 15, they are directed to a system or apparatuses to implement the method of steps set forth in claims 5, 6, and 8. Matl teaches the claimed method of steps in claims 5, 6, and 8. Therefore, Matl teaches the system or apparatuses, to implement the claimed method of steps, in claims 12, 13, and 15.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Matl in view of Park (PG Pub. No. 20230205147), herein “Park.”
Regarding claim 7,
The previously cited reference(s) teach the limitations of claim 1 which claim 7 depends. Matl does not teach controlling an accessory (robot and robotic arm) on a conveyor or rail. However, Park does teach that the adjustment of nest offset is performed by controlling an accessory on a pallet of the conveyor system. (Par. 0020: “…the controller may control a movement of the transfer robot such that the transfer robot performs a loading or unloading operation of the substrate in/from the slots of the transfer target object along the position coordinates for each slot.” Park shows in figure 5 an accessory on a pallet (base item 810) that moves on a conveyor (rail 143). See also claim 8 or Park.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have combined the method and system of dynamic adjustment of a control element of a conveyor system wherein the system uses an analysis process to determine bin positioning as in Matl with control the robotic device that is positioned on a pallet (body or base) that moves along a conveyor system (rail) as in Park in order to have a method of have transfer equipment be capable of precise teaching through vision processing by a rail system (Par. 0005)
Claims 11 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Matl in view of Nguyen et al. (PG Pub. No. 20210064937), herein “Nguyen.”
Regarding claim 11,
The previously cited reference(s) teach the limitations of claim 9 which claim 11 depends. Matl does not teach updating a ML model based on an unavailable or ML model does not return a result. However, Nguyen does teach that the ML module comprises a current ML model and the ML module is configured to request further input if the current ML model doesn’t return a result and update the ML model based on the further input. (Par. 0027: “As an example, field user 53 may be performing an action (e.g., operating, maintaining) on an industrial device 62 (e.g., an industrial automation component), such as a controller, a drive, a motor, a sensor, a conveyor, an input/output (I/O) module, a motor control center…” Par. 0023: “For example, the training content may include recorded steps based on instructions (e.g., verbal guidelines) provided by the remote expert and a response provided by the field user indicating a completed step, as performed via the training system. Such training content may be stored and may be retrieved from a database or other suitable storage component at a later time, so as to instruct a future field user to perform a similar or the same task. Additionally, the training system may automatically detect a variant or unexpected response or feedback provided by the future field user indicating a particular step could not be completed, such as due to an unclear instruction. The training system may use an artificial intelligence component or machine learning algorithms to update the training content to help another future field user to complete the respective task with clearer instructions or details. As such, the training system may automatically generate and adjust training content to assist future field users in performing the particular task.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have combined the method and system of dynamic adjustment of a control element of a conveyor system wherein the system uses an analysis process to determine bin positioning as in Matl with an industrial system and method that consists of a training system wherein a conveyor in a factor may be controlled wherein machine learning is used and the machine learning is updated when part of the training system feedback could not be completed as in Nguyen in order to adjust training content to assist in performing a particular task to an industrial system.
Regarding claim 4, it is directed to a method to implement the system set forth in claim 11. Matl and Nguyen teach the system of claim 11. Therefore, Matl and Nguyen teach the method of steps in to implement the system or apparatuses in claim 4.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Stubbs et al. (US PG Pub. No. 20170080566) is related to the instant application and teaches robotic manipulators that are coupled with a convey system/belt (item 130). Paragraph 0074 also teaches analyzing manipulation data of the robot stations and uses machine learning algorithm to collected data used to effect the use of a robot arm (item 110) on the conveyor.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAD G ERDMAN whose telephone number is (571)270-0177. The examiner can normally be reached Mon - Fri 7am - 5pm EST; Off every other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamini S. Shah can be reached at (571) 272-2279. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHAD G ERDMAN/Primary Examiner, Art Unit 2116