Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-9, and 11-18 are rejected under 35 U.S.C. 103 as being unpatentable over Khodl (US 2022/0177227) in view of Das (US 2017/0140791).
As per claim 1, and 17-18 Khodl teaches, a method and apparatus and a computer software product, the product comprising a non-transitory computer-readable medium, comprising: collecting a set of overlapping video segments that cover a warehouse storing multiple items in respective bins (Khodl, fig.3 showing the bins and video cameras 14 in warehouse multiple items in respective bins and ¶[0024] “The optical device 14 may include various functionalities including video capability”); generate image sequence in a coordinate system of the warehouse (Khodl, ¶[009] “he control system is configured to determine location coordinates of the particular location within an accuracy of about 1 millimeter to about 3 millimeters relative to a pre-determined desired location.” The video camera 14 would have this capability to be able to find the location via coordinates of the warehouse); identifying multiple individuals performing picking actions from different ones of the bins at respective coordinates in the coordinate system (Khodl, fig.2 multiple individuals 13s being identified performing the picking operations in the bins); computing, based on the merged video image sequence, respective coordinates in the coordinate system of the bins from which the picking actions were performed (Khodl, ¶ [0010] “In another form of the present invention, a method and is provided for monitoring a location of a first object in relation to another object to update a coordinate database based on the relative location of the first object.” Based on the location of the other object then represents merge video images to be able to compare the two objected ); retrieving, from a warehouse management system, a set of first work orders, each of the first work orders performed by a given individual and comprising one or more of the items (Khodl, ¶[0011] “The first object may include an operator performing functions at an order fulfillment workstation, a storage rack of an automated storage and retrieval system, or an AGV, for example.” This would represent worker 13 getting work orders and ¶[0012] “Accordingly, the method and system of the present invention provide an order fulfillment workstation to track physical placement of items, monitor the accuracy and completion of pick/put operations, and track productivity of the operator and/or the warehouse/fulfillment facility.” ); analyzing, by a processor, the picking actions, the coordinates of the bins, and the first work orders so as to establish a correspondence between the bins and the items (Khodl, ¶[0012] “The method and system are particularly useful for tracking placement and retrieval of items within an order fulfilment workstation, and may also provide real-time, relative coordinate information of infrastructure within a warehouse.” This would represent analyzing the picking actions, the coordinates of the bins, and the first work orders so as to establish a correspondence between the bins and the items by monitoring the warehouse and keeping track of relative coordinate information and ¶[0022] “Referring now to the drawings and the illustrative embodiments depicted therein, a pick tracking method 100 and a location monitoring system, in the form of a pick tracking/tracker system 10, are provided for tracking and verifying a pick or put of an item at a particular location within an order fulfilment workstation 12, such as at a receptacle or bin (FIGS. 1 and 2).”); and applying the correspondence in verifying execution of second work orders performed subsequent to performance of the set of first work orders (Khodl, fig.8 210 update coordinate database based on relative location on indicia, and fig.1 112-114 as the system loops around there would be second work orders and different picking tasks).
Khodl does not clearly teach, stitching together the video segments so as to generate a merged video and identifying, in the merged video image sequence.
Das teaches, stitching together the video segments so as to generate a merged video and identifying, in the merged video image sequence (Das, ¶ [0022] In a second example, the views of the cameras that are stitched together overlap. There is an overlap between each camera in the right edge of one camera's image and the left edge of the adjacent camera's image. In these overlap areas, the images are the same except for parallax and other effects as noted above. When the images are stitched together, the seam may be placed anywhere within the overlapping fields of view. “ this represents stitching together video to generated merged video and identifying the merge video ).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Das to be put in the Khodl camera 14 as an added feature.
The motivation would have been to improve productivity.
As per claim 2, Khodl in view of Das teaches, the method according to claim 1, wherein the video cameras have respective fields of view (FOV), and where the fields of view of pairs of the cameras that are adjacent to one another have an overlap between 10% and 30% (Khodl, fig.5 14 showing overlap and would have such overlaps).
As per claim 5, Khodl in view of Das teaches, the method according to claim 1, wherein a given picking action comprises retrieving one or more given items (Khodl, ¶[0022] “The system 10 is operable to track physical placement for item locations, verify order picking accuracy, and track productivity of the operator and/or the warehouse or fulfillment facility. The method 100 and system 10 are operable to determine placement of items at a high degree of accuracy, thereby enabling verification of item placement at a particular location within a receptacle (e.g. within one to three millimeters (1 mm-3 mm) of the desired location in the receptacle). The method 100 is particularly useful for item/piece picking operations as well as case/pallet picking operations.” And fig.2 13 represents item picking).
As per claim 6, Khodl in view of Das teaches, the method according to claim 1, wherein a given picking action comprises restocking one or more given items (Khodl, ¶[0023] “The operator 13 performs 104 a pick or put operation, such as picking or putting an item to a particular location in the order fulfilment workstation 12. The method 100 includes capturing image data 106,” restocking is being represented by the put options).
As per claim 7, Khodl in view of Das teaches, the method according to claim 1, wherein a given picking action comprises dropping one or more given items (Khodl, ¶[0023] “The operator 13 performs 104 a pick or put operation, such as picking or putting an item to a particular location in the order fulfilment workstation 12. The method 100 includes capturing image data 106,” The put action is dropping one more given items).
As per claim 8, Khodl in view of Das teaches, the method according to claim 1, wherein computing the respective coordinates of the bins comprises generating a heat map of the coordinates of the picking action, and detecting, clusters of the coordinates in the heat map, wherein the clusters correspond to the bins (Khodl, ¶[0024] “The optical device 14 may include various functionalities including video capability, infrared capability, etc.” infrared capability represents having a heat map).
As per claim 9, Khodl in view of Das teaches, the method according to claim 8, wherein generating the heat map comprises applying a clustering algorithm to the coordinates of the picking action so as to generate the detected clusters (Khodl, ¶[0024] “The optical device 14 may include various functionalities including video capability, infrared capability, etc.” the clusters would be a infrared view).
As per claim 11, Khodl in view of Das teaches, the method according to claim 1, wherein establishing the correspondence between the bins and the items comprises applying a majority voting algorithm to the picking actions, the coordinates of the bins, and the first work orders (Khodl, fig.1 112. This determination would represent majority voting rule to be able to see if an item is in the bin or not ).
As per claim 12, Khodl in view of Das teaches, the method according to claim 1, wherein verifying execution of a given second work order comprises generating an alert upon detecting, based on the established correspondence, a picking error for a given item in the work order (Khodl, fig.1 114 verify the order is correct if not another job would then be given).
As per claim 13, Khodl in view of Das teaches, the method according to claim 12, and further comprising updating the corresponding bin for the given item upon receiving an override for the alert (Khodl, fig.1 112 verify would be the override over the alert then the items are placed right).
As per claim 14, Khodl in view of Das teaches, the method according to claim 1, wherein the merged video image sequence covers all the bins (Khodl, fig.3 as can be seen all bins are covered by cameras 14).
As per claim 15, Khodl in view of Das teaches, the method according to claim 1, wherein a given bin comprises a workstation configured to generate the work orders ( Khodl, ¶[0012] “Accordingly, the method and system of the present invention provide an order fulfillment workstation to track physical placement of items, monitor the accuracy and completion of pick/put operations, and track productivity of the operator and/or the warehouse/fulfillment facility.” This represents the workstation configured to generate work orders).
As per claim 16, Khodl in view of Das teaches, the method according to claim 15, wherein a given picking action comprises retrieving a given work order from the workstation ( Khodl, ¶[0012] “Accordingly, the method and system of the present invention provide an order fulfillment workstation to track physical placement of items, monitor the accuracy and completion of pick/put operations, and track productivity of the operator and/or the warehouse/fulfillment facility.” The instructions for the work come from this workstation).
Allowable Subject Matter
Claims 3, 4 and 10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SANTIAGO GARCIA whose telephone number is (571)270-5182. The examiner can normally be reached Monday-Friday 9:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SANTIAGO GARCIA/Primary Examiner, Art Unit 2673
/SG/
Prosecution Insights