Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-8, and13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Welker et al. (US 20200139407 A1) in view of Cohn (US 5813542 A).
Regarding claim 1, Welker et al. teaches a method for sorting and/or treating containers (see para [0002]; “an apparatus for sorting containers and to a method for sorting containers”), comprising the steps of: - recording at least one image and/or a video of a plurality of containers by an image recording device (see para [0011]; “the identification device comprises at least one camera device for optically detecting the decor of the containers”), analyzing the at least one recorded image (see para [0011]; “by means of an evaluation device, it is determined whether the specifically detected decor belongs to a first class of containers or to a second class of containers”); - identifying the individual containers (see para [0011]; “a clear assignment between a specific container and a specific shifting means is created”); - assigning an identification information and at least one portion of the recorded image to each of the identified containers (see para [0011]; “an assignment of the position of the respective container to be transferred from the main path to the secondary path and of the position of a specific shifting means can be generated”). However, Welker et al. does not teach which is configured for recording spatially resolved color images; - and - determining a color information, which is characteristic of an identified container, from the portion of the recorded image.
In the same field of endeavor, Cohn teach which is configured for recording spatially resolved color images (see Abstract; “sensing a multiple color image of at least a portion of the object and producing color signals indicative of a plurality of colors in response to sensing the multiple color image”); - and - determining a color information, which is characteristic of an identified container, from the portion of the recorded image (see Abstract; “The color signals are transformed to a hue signal and a saturation signal, and the object is classified in response to the hue signal and the saturation signa”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effecting filling date of the invention to modify an apparatus for sorting containers of Welker et al. in view of a method for a color sorting system of Cohn in order to maximized, the speed and number of objects sorted while simultaneously minimizing errors from rotational movement of objects between cameras (see Abstract).
Regarding claim 2, the rejection of claim 1 is incorporated herein.
Welker et al. in the combination further teach wherein an actuator device and/or sorting device, which is configured for acting on the identified container (see para [0029]; “Preferably, the shifting means 14 move the containers from a first conveyor belt 6 to a second conveyor belt 61.. ..The shifting movement of the shifting means 14 is preferably effectuated by means of actuators. Preferably, each shifting means 14 comprises such an actuator, in particular, a lifting element or shifting element”).
Cohn in the combination further teach is controlled taking into account the color information (see col. 9, lines 29-38; “The control section 326 is used to store, retrieve, disable, and enable different predefined patterns on the color wheel 302. Further, a set of patterns can be used for multiple lanes (sort channels) of products in order allow simultaneous sorting of multiple different types of objects, each with a different classification criteria. The color sorter also includes a capture facility whereby an image of an object can be captured on the display and its color content displayed on the color wheel to assist the operator in defining that object as acceptable or rejectable”).
Regarding claim 3, the rejection of claim 2 is incorporated herein.
Welker et al. in the combination further teach wherein the actuator device is selected from a group of actuator devices, which contains robots, robot arms, impact devices for ejecting individual containers from the transport path, switches for discharging individual containers from the transport path, or the like (see para [0029]; “the shifting means 14 move the containers from a first conveyor belt 6 to a second conveyor belt 61. The first conveyor belt 6 preferably moves along the main transport path 18 and the second conveyor belt 61 preferably moves along the secondary transport path 20. Preferably, these conveyor belts always move at the same speed, in particular, in the selection region 16. The shifting movement of the shifting means 14 is preferably effectuated by means of actuators. Preferably, each shifting means 14 comprises such an actuator, in particular, a lifting element or shifting element”).
Regarding claim 4, the rejection of claim 1 is incorporated herein.
Welker et al. in the combination further teach wherein a transport device transports the containers along a predetermined transport path, and the at least one image or video of the containers is recorded during the transport of the containers (see para [0011]; “by means of the conveyor belt, at least one set of containers on a main transport path of the identification device can be supplied, wherein the identification device comprises at least one camera device for optically detecting the decor of the containers”).
Regarding claim 5, the rejection of claim 1 is incorporated herein.
Cohn in the combination further teach wherein the at least one image is analyzed by an image recognition algorithm, and/or the individual containers or container regions are identified by an algorithm, and/or the color information is determined by an algorithm (see col. 7, lines 30-32; “both an HSI color model and one algorithm to obtain the HSI color model from a RBG color model, and is incorporated herein by reference”).
Regarding claim 6, the rejection of claim 1 is incorporated herein.
Welker et al. in the combination further teach wherein all containers in a recorded image are identified (see para [0011]; “at least one set of containers on a main transport path of the identification device can be supplied, wherein the identification device comprises at least one camera device for optically detecting the decor of the containers”).
Regarding claim 7, the rejection of claim 1 is incorporated herein.
Welker et al. in the combination further teach wherein a container in camera coordinates is found by the identification information, and/or the identification information is transferred to a superordinate coordinate system (see para [0011]; “an assignment of the position of the respective container to be transferred from the main path to the secondary path and of the position of a specific shifting means can be generated”).
Regarding claim 8, the rejection of claim 1 is incorporated herein.
Cohn in the combination further teach wherein a particular color or color group is assigned to each container or container type (see col. 3, lines 7-13; “a memory contains data representative of the hue and saturation values, and the classification of the object is based on a comparison of the hue signal and the saturation signal to the data. By classifying the object in response to the hue signal and the saturation signal, more accurate color recognition can be made in order to properly classify an object”).
Regarding claim 13, Welker et al. teaches an apparatus for sorting and/or identifying containers (see para [0002]; “an apparatus for sorting containers and to a method for sorting containers”), wherein the apparatus has a transport device configured to transport the containers along a predetermined transport path (see para [0011]; “by means of the conveyor belt, at least one set of containers on a main transport path of the identification device can be supplied, wherein the identification device comprises at least one camera device for optically detecting the decor of the containers”), and wherein the apparatus has at least one image recording device configured for recording at least one image and/or a video of a plurality of containers transported by the transport device (see para [0011]; “at least one set of containers on a main transport path of the identification device can be supplied, wherein the identification device comprises at least one camera device for optically detecting the decor of the containers”), wherein the apparatus furthermore comprises an analysis device configured for analyzing the recorded image (see para [0011]; “by means of an evaluation device, it is determined whether the specifically detected decor belongs to a first class of containers or to a second class of containers”), which analysis device is configured for identifying an individual container within the recorded image (see para [0011]; “a clear assignment between a specific container and a specific shifting means is created”), wherein the apparatus has a first assignment device, which is configured for assigning identification information to an image portion, containing the identified container, of the recorded image (see para [0011]; “an assignment of the position of the respective container to be transferred from the main path to the secondary path and of the position of a specific shifting means can be generated”). However, Welker et al. does not teach and wherein the image recording device is configured for recording spatially resolved color images, and a color information determination device, which is configured for determining color information characteristic of this container and/or of this image portion.
In the same field of endeavor, Cohn teach and wherein the image recording device is configured for recording spatially resolved color images, and a color information determination device, which is configured for determining color information characteristic of this container and/or of this image portion (see Abstract; “A method of classifying objects comprises the steps of sensing a multiple color image of at least a portion of the object and producing color signals indicative of a plurality of colors in response to sensing the multiple color image. The color signals are transformed to a hue signal and a saturation signal, and the object is classified in response to the hue signal and the saturation signal”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effecting filling date of the invention to modify an apparatus for sorting containers of Welker et al. in view of a method for a color sorting system of Cohn in order to maximized the speed and number of objects sorted while simultaneously minimizing errors from rotational movement of objects between cameras (see Abstract).
Regarding claim 14, the rejection of claim 13 is incorporated herein.
Welker et al. in the combination further teach wherein the apparatus has an actuator device configured for acting on the containers (see para [0014]; “each shifting means can be shifted by means of each at least one actuator in a direction orientated in an inclined manner towards the transport direction of the conveyor belt, wherein the actuator is an integral component of the selection device and can be moved along with the shifting means along the continuous track”), wherein a control device is furthermore provided (see para [0025]; “the evaluation device, generates data for controlling the transport device 10. So that shifting means 14 are moved along the continuous track 12 in such a way that they move at least in the transport direction T at the same speed as the containers 2, 4 transported on the conveyor belt 6”).
Cohn in the combination further teaches which controls this actuator device as a function of the color information characteristic of the container (see col. 9, lines; “The control section 326 is used to store, retrieve, disable, and enable different predefined patterns on the color wheel 302. Further, a set of patterns can be used for multiple lanes (sort channels) of products in order allow simultaneous sorting of multiple different types of objects, each with a different classification criteria. The color sorter also includes a capture facility whereby an image of an object can be captured on the display and its color content displayed on the color wheel to assist the operator in defining that object as acceptable or rejectable”).
Regarding claim 15, the rejection of claim 13 is incorporated herein.
Welker et al. in the combination further teach wherein the apparatus has a second assignment device (see para [0011]; “an assignment of the position of the respective container to be transferred from the main path to the secondary path and of the position of a specific shifting means can be generated. This embodiment is favorable since a clear assignment between a specific container and a specific shifting means is created”), configured to assign a container type (see para [0011]; “an evaluation device, it is determined whether the specifically detected decor belongs to a first class of containers or to a second class of containers”).
Cohn in the combination further teaches to the color information characteristic of the image portion and/or the container (see Abstract; “the object is classified in response to the hue signal and the saturation signal”).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Welker et al. in view of Cohn as applied in claim 1 above, and further in view of Connolly (US 5432545 A).
Regarding claim 9, the rejection of claim 1 is incorporated herein. The combination of Welker et al. and Cohn as a whole does not teach wherein at least one color is found in the image portion and the container is deduced on the basis of proportions of the individual colors.
In the same field of endeavor, Connolly teaches wherein at least one color is found in the image portion and the container is deduced on the basis of proportions of the individual colors (see Abstract; “A method is provided for detecting the color of objects such as bottles to be recycled so as to enable the bottles to be separated by color”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effecting filling date of the invention to modify an apparatus for sorting containers of Welker et al. in view of a method for a color sorting system of Cohn and further in view of color detection and separation method of Connolly in order to determine the color of an object having a label of a different color attached thereto (see Abstract).
Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Welker et al. in view of Cohn as applied in claim 1 above, and further in view of Ding et al. (US 20210027497 A1).
Regarding claim 10, the rejection of claim 9 is incorporated herein. The combination of Welker et al. and Cohn as a whole does not teach wherein the colors are found using at least one color system, wherein the color system is selected from a group of color systems that contains HSV, L*A*B, and YCbCr.
In the same field of endeavor, Ding et al. teaches wherein the colors are found using at least one color system, wherein the color system is selected from a group of color systems that contains HSV, L*A*B, and YCbCr (see para [0046]; “Another color model is the CIELAB (or simply “LAB”) that expresses three components for a color—L* for the lightness from black (0) to white (100), a* from green (−) to red (+), and b* from blue (−) to yellow (+). Still, other color models include the HSL (hue, saturation, lightness) model, HSV (hue, saturation, value) model, and CMYK (cyan, magenta, yellow, key)”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effecting filling date of the invention to modify an apparatus for sorting containers of Welker et al. in view of a method for a color sorting system of Cohn and further in view of classifying colors of objects in digital images of Ding et al. in order to utilize an improved color classifier model that accurately and efficiently classifies the color of objects in images (see Abstract).
Regarding claim 11, the rejection of claim 1 is incorporated herein.
Ding et al. in the combination further teach wherein the analysis of the at least one recorded image, the identification of the individual containers, and/or the determination of the color information takes place using an artificial intelligence (see para [0051]; “above, the color classification system can employ machine learning and various neural networks in various embodiments…… Machine learning can include neural networks (e.g., an object detection neural network, an object mask neural network, and/or an object classification neural network), data-based models, or a combination of networks and models”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effecting filling date of the invention to modify an apparatus for sorting containers of Welker et al. in view of a method for a color sorting system of Cohn and further in view of classifying colors of objects in digital images of Ding et al. in order to utilize an improved color classifier model that accurately and efficiently classifies the color of objects in images (see Abstract).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Welker et al. and Cohn in view of Connolly as applied in claims 1, and 9 above, and further in view of Ding et al.
Regarding claim 12, the rejection of claim 9 is incorporated herein. The combination of Welker et al. and Cohn as a whole does not teach wherein a determined color structure of an image portion is assigned to a particular container or a particular container type.
In the same field of endeavor, Ding et al. teach wherein a determined color structure of an image portion is assigned to a particular container or a particular container type (see para [0009]; “For pixels representing the object, the disclosed systems can map the pixels to the multidimensional color space to determine a color correspondence to the color similarity region (or regions for the multiple colors). … generate a color-matching score for the object with respect to the color based on the color correspondences between the object pixels and the color similarity region. Further, the disclosed systems can classify the object as the color based on the color-matching score”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effecting filling date of the invention to modify an apparatus for sorting containers of Welker et al. in view of a method for a color sorting system of Cohn and color detection and separation method of Connolly and further in view of classifying colors of objects in digital images of Ding et al. in order to satisfying a minimum color-matching threshold for the color (see Abstract).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WINTA GEBRESLASSIE whose telephone number is (571)272-3475. The examiner can normally be reached Monday-Friday9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WINTA GEBRESLASSIE/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677