DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see remarks, filed 01/30/2026, with respect to the rejection(s) of claim(s) 112-126 under Kalouche and/or Kalouche/Chamberlin have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Kalouche and/or Kalouche/Chamberlin in view of Van Durrett US 5,501,571.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 112-123, 125, 142 and 143 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kalouche US 2021/0032034 in view of Van Durrett 5,501,571.
Kalouche discloses a system for handling a plurality of objects, comprising:
(Re claim 112) “a robotic arm configured to pick two or more objects of said plurality of objects from a first location and place each object of said two or more objects, said robotic arm comprising” (206 figure 9A). “(i) at least one end effector receiver configured to receive at least one end effector” (232 figure 13A). “(ii) an end effector stage comprising two or more end effectors” (260 figure 9A,9F,9G). “at least one optical sensor configured to obtain first image data of a first object of said two or more object and second image data of a second object of said two or more objects” (262, 264 figure 9B). “a computing device comprising:(i) a processor operatively coupled to said robotic arm and said at least one optical sensor” (abstract). “(ii) one or more non-transitory computer readable storage media with a computer program including instructions, that when executed by said processor, cause said processor to perform operations comprising: analyzing said first image data of said first object to select said at least one end effector from said two or more end effectors” (abstract, ‘gripping tool 248 may be selected based upon … analysis of the image data’ para 0142). “causing said robotic arm to, using said at least one end effector, pick said first object from said first location and place said first object at a second location” (‘inform the picking arm as to which order bin 214 the product should be released’ para 0144). “applying a trained machine learning model to said second image data of said second object to detect … with said second object” (‘machine learning … explicitly analyze images’ para 0145).
Kalouche does not disclose identifying an anomaly, in response to detecting said anomaly associated with said second object automatically causing said robot arm to pick said second object from said first location and place said second object at and exception location and generating an alert accessible to an operator via a computing device, wherein said alert comprises an indication that said second object was placed at said exception location.
Van Durrett teaches identifying an anomaly (col 3 lines 12-27), in response to detecting said anomaly associated with said second object automatically causing said robot system to allow said second object from said first location and place said second object at and exception location (‘diverted … reject carton station’ col 3 lines 12-27) and generating an alert accessible to an operator via a computing device, wherein said alert comprises an indication that said second object was placed at said exception location (‘exception report for reject cartons’ col 7 lines 36-41).
It would have been obvious to one skilled in the art to modify the system of Kalouche to include using its machine learning image analysis to identify an anomaly, because it helps prevent the wrong or damaged item from being placed with an order; including that in response to detecting said anomaly associated with said second object automatically causing said robot arm to pick said second object from said first location and place said second object at and exception location, because it helps remove damaged or incorrect items from the system; and to include generating an alert accessible to an operator via a computing device, wherein said alert comprises an indication that said second object was placed at said exception location, because it helps the operator to know that there have been anomalous items identified and this may have effected filling of the order.
(Re claim 113) “read a machine-readable code marked on said second object” (262, 264 figure 9B, para 0144).
(Re claim 114) “said anomaly is detected based at least in part on said machine-readable code being different than one or more expected machine-readable codes” (‘confirming that the correct product has been grasped’ para 0144).
(Re claim 115) “a product database in communication with said computing device, wherein said product database provides said one or more expected machine-readable codes” (‘confirming that the correct product has been grasped’ para 0144). Confirming that the code is correct would require a database of codes for comparison.
(Re claim 116) Kalouche discloses the system as rejected above and “(i) analyze said second image data to obtain one or more measured dimensions of at least one of said one or more objects” (para 0128).
Kalouche does not disclose (ii) generate an alert if a difference between said one or more measured dimensions and one or more expected dimensions of said at least one of said one or more objects exceeds a predetermined threshold.
Van Durrett teaches (ii) wherein said anomaly is detected based at least in part on a difference between said one or more measured dimensions of said second object and one or more expected dimensions of said second object exceeds a predetermined threshold ( ‘a large deviation from the size expected’ col 7 lines 1-10, col 3 lines 12-14).
It would have been obvious to one skilled in the art to modify the system of Kalouche to include wherein said anomaly is detected based at least in part on a difference between said one or more measured dimensions of said second object and one or more expected dimensions of said second object exceeds a predetermined threshold because it can help identify incorrect picks or damage items.
(Re claim 117) “read a machine-readable code marked on said second object to obtain said one or more expected dimensions of said second object” (‘size, shape, material or weight’ para 0126, ‘barcode’ para 0144).
(Re claim 118) “instructions, when executed by said processor, further cause said processor to instruct said robotic arm to present said machine-readable code to said at least one optical sensor, such that said at least one optical sensor is able to scan said machine-readable code” (‘wave/rotate the product in front of scanners’ para 0144).
(Re claim 119) “a product database in communication with said computing device, wherein said product database comprises said one or more expected dimensions of said second object” (‘size, shape, material or weight’ para 0126).
(Re claim 120) “an operator device, wherein said instructions, when executed by said processor, further cause said processor to send alert information to said operator device when said alert is generated” (‘notification to operator interface’ para 0137).
(Re claim 121). “said alert information comprises one or more images of said second object” (‘images and/or video’ para 0075).
(Re claim 122) “said operator device comprises a user interface for receiving input from an operator” (‘operator interface 102’ para 0075).
(Re claim 123) “trained machine learning model is trained via operations” (‘train the machine learning system’ para 0079). “applying a machine learning model to an image of an object to generate a prediction of whether said object corresponds to a certain anomaly, receiving a verification at an interface that said prediction is incorrect; and adjusting a threshold based at least in part on said input, wherein said threshold corresponds to a difference between a measured property of said object and an expected property of said object, there by obtaining a trained machine learning model” The claimed steps are a general method of training a machine learning model such as the one used by Kalouche; and Van Durrett discloses that the anomaly detection can include a deviation in expect size.
(Re claim 125) “said processor of said computing device is operatively coupled to said at least one optical sensor, and wherein said instructions, when executed by said processor, further cause said processor to analyze said first image data of said first object to obtain one or more grasping points on said first object for said at least one end effector” (‘grasping pose’ para 0102).
(Re claim 142) “said second object is an unboxed item” (figure 9D, 9E).
(Re claim 143) Kalouche does not disclose that said anomaly is associated with said second object is damage to the unboxed item.
Van Durrett discloses that said anomaly is associated with said second object is damage to the item (col 3 lines 12-13).
It would have been obvious to one skilled in the art to modify the system of Kalouche to include that said anomaly is associated with said second object is damage to the item because it helps prevent damaged items from being included in the order.
Claim(s) 126 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kalouche/Van Durrett in view of Chamberlin US 10,810,715.
(Re claim 126) Kalouche/Van Durrett discloses the system as rejected above.
Kalouche/Van Durrett does not disclose at least one force sensor configured to obtain a measured force of said second object from said at least one effector, and wherein said anomaly is detected based at least in part on a force differential of said measured force of said second object and an expected force of said second object.
Chamberlan teaches at least one force sensor configured to obtain a measured force of said second object from said at least one effector, and wherein said anomaly is detected based at least in part on a force differential of said measured force of said second object and an expected force of said second object (‘pick-error alert … expected dimension’ claim 1, claim 3).
It would have been obvious to one skilled in the art to modify the system of Kalouche/Van Durrett to include at least one force sensor configured to obtain a measured force of said second object from said at least one effector, and wherein said anomaly is detected based at least in part on a force differential of said measured force of said second object and an expected force of said second object because it can help identify incorrect picks.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 9,996,890 discloses a machine learning system where the dimension thresholds are modified.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIMOTHY R WAGGONER whose telephone number is (571)272-8204. The examiner can normally be reached Mon-Thurs 5am-330pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jacob Scott can be reached at 571-270-3415. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
TIMOTHY R. WAGGONER
Primary Examiner
Art Unit 3655 B
/TIMOTHY R WAGGONER/Primary Examiner, Art Unit 3655