Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claim 21 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention:
applicant specification does not specifically discuss that stacking of items explicitly results in occlusion of labels. For instance, the word “occlusion” is not recited in Applicant original disclosure.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
A. Claims 1-2, 5,-6, 8-13, 15-17, 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Kanezaki (RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints, CVPR 2018) in view of Bailey (US 20110046775) in further view of Jefkine (Backpropagation In Convolutional Neural Networks, DeepGrid, https://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/ available 2016)
As for claim 1, Kanezaki teaches
A system for processing items, the system comprising:
a memory storing a model for recognizing information on an item regardless of item orientation or capturing environment, the model trained based at least in part on
a training data set of photographs taken of labels
at different positions (Kanezaki, Fig 2, images of same object from different view angles)
different lighting conditions with respect to an imaging device (“different lighting conditions with respect to an imaging device” does not indicate what must cause the lighting conditions to be different, thus either any change introduced in the capturing environment is reasonably interpreted as a change causing a different lighting on the object being captured; Kanezaki, Fig 1, see image A of a chair with the chair back illuminated with greater brightness than in image B; while Kanezaki is silent on the illumination source being moved or adjusted in brightness, the claim does not explicitly require this)
in different capturing environments possible in a processing facility (Kanezaki, Fig 3, ch 3, p 5013 par “Viewpoint setups for training”, cases (i) through (iii), teaches multiple camera setups)
Kanezaki does not explicitly teach, Bailey however teaches
a scanner configured to capture an image of a plurality of items; (Bailey [0401] camera or barcode scanner) in a processing facility ([0009] a mail sorting facility)
one or more processors in data communication with the scanner and the memory and configured to run the model on the captured image to identify information on the at least one of the plurality of first items ([0370] processor+memory embodiment)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the object recognition system of Kanezaki, by further including features of parcel sorting of Bailey, as both pertain to reading information from target objects. The motivation to do so would have been, to enhance recognition of parcel categories as taught by Bailey.
The combination of Kanezaki and Bailey does not specifically teach, Jefkine however teaches
storing the recognized information as an obtained model output in the memory; (Jefkine, pg 6, eq (9), the predicted model output yp must be inherently stored in some kind of memory in order to be inputted into the calculation of equation (9) of Jefkine)
calculating a difference between the obtained model output and an expected model output to generate a weight value adjustment for a node of the model (equation (9), the difference “tp – yp” which is the difference between the output obtained during training and the target ground-truth output); and
applying the weight value adjustment to the node of the model. (description of equation (9), “learning will be achieved by adjusting the weights.”)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the combination of Kanezaki and Bailey by further including the specifics of neural network training of Jefkine, as all pertain to training convolutional neural networks. The motivation to do so would have been: Kanezaki, p 5013 left col eq (4) description, teaches performing standard back-propagation but does not provide details of this already well-known technique. Jefkine teaches the technique of backpropagation and adjusting the weights based on the difference between the output obtained during training and the target ground-truth output.
As for independent claim 11, please see discussion of analogous claim 1 above.
As for claims 2, 13, the combination of Kanezaki and Bailey teaches
the plurality of items is a mail item, and wherein the information comprises an address, a sender, a recipient, a barcode, or postage indicia (Bailey [0468] identification of parcel by barcode)
As for claims 5, 15, the combination of Kanezaki and Bailey teaches
identify a label type on each of the plurality of items in the captured image (Bailey [0468] identification of parcel by barcode)
As for claims 6, 16, the combination of Kanezaki and Bailey teaches
store, in the memory, a status of the at least one of the plurality of items (see claim 5 above, it is inherent that the information contained in the barcode, after being read by the system, must be stored at least temporarily, to be used in subsequent steps)
As for claims 8, the combination of Kanezaki and Bailey teaches
a robotic arm configured to move one or more of the plurality of first items (Bailey [1616] ln 17-20 “a robotic arm”)
As for claims 9, 17, the combination of Kanezaki and Bailey teaches
the one or more processors is further configured to, based on the recognized information on one of the plurality of itmes, control the robotic arm automatically to move the one of the plurality of items identified for special treatment (Bailey Fig 1E [0399] parcels are sorted by category into different destination areas)
As for claims 10, 19, the combination of Kanezaki and Bailey teaches
the scanner is further configured to capture an image of at least one of the plurality of items (as discussed in claim 1), and wherein the one or more processors are configured to:
identify a service class indicator on the label on one of the plurality of items; and cause item processing equipment to move the item to a sort location based on the identified service class indication (Bailey Fig 1E [0399] parcels are sorted by category into different destination areas)
As for claim 12, the combination of Kanezaki and Bailey teaches
one or more of the plurality of first items are at least partially stacked on each other, and wherein capturing the image of at least one item of a plurality of first items comprises capturing an image of the one or more plurality of first items which are at least partially stacked on each other (Bailey [00421] receiving stacked mail pieces)
As for claim 20, the combination of Kanezaki and Bailey teaches
A non-transitory computer readable recording medium storing instructions, when executed by one or more processors, cause the one or more processors to perform the method of claim 11 (Bailey [0065] software embodiment)
As for claim 21, the combination of Kanezaki and Bailey teaches
the training data set of photographs further comprises photographs of labels taken in different item stacking conditions that result in partial occlusion of labels (please see rejection 112.1, applicant specification does not specifically discuss that stacking of items explicitly results in occlusion of labels; however it can be implicitly understood that as items are stacked, labels on some objects may be occluded by other objects; in light of that interpretation, Bailey [0042-0043] also discusses that mail pieces may be stacked, therefore implicitly creating occlusions for the same reasons)
B. Claims 3, 7, 14 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kanezaki in view of Bailey in further view of Code7700 (“Hazardous Materials (HazMat)”, updated 2016, retrieved from https://code7700.com/hazmat.htm)
As for claims 3, 14, the combination of Kanezaki and Bailey does not explicitly teach, Code7700 however teaches
each of the plurality of items has a label disposed thereon, wherein the information is contained on the label, and wherein the information comprises a service class indicator, a special treatment label, a warning label, or a hazard label (Code7700 pg 4, illustrates various labels for hazardous materials and other warnings)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the combination of Kazenaki and Bailey, by including recognition of warning labels on the parcel, as all pertain to reading information on target objects. The motivation to do so would have been, to allow for separate handling of packages with specified risk.
As for claims 7, 18, the combination of Kanezaki, Bailey and Code7700 however teaches
in response to the identified label type being a warning label or hazard label, the one or more processors are configured to associate the one of the plurality of items having a warning label or hazard label thereon with a special handling instruction (Code7700 pg 2-3, “Will not carry” rationale – discusses carrying vs not carrying various objects according to the warning labels)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the combination of Kazenaki and Baley, by including the selection step of carrying or not carrying a particular package according to the warning label. The motivation to do so would have been, to prevent accidents when an operator is not authorized to carry packages with a particular warning label.
Final Rejection
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK ROZ whose telephone number is (571)270-3382. The examiner can normally be reached on 9AM-5PM M-F.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached on (571)272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARK ROZ/
Primary Examiner, Art Unit 2669