Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 04, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2 and 4 – 7 are rejected under 35 U.S.C 103 as being unpatentable over Dal Mutto et al. US Patent Publication No. US-20190108396-A1 (hereinafter Carlo) in view of BACK US Patent Application Publication No. US-20230289971-A1 (hereinafter Seunghyeok) and further in view of Von Einem US Patent Application Publication No. US-20230230399-A1 (hereinafter Volker).
Regarding claim 1, Carlo discloses a method for determining at least one container information (coi) about a laboratory sample container (Carlo in [0005] discloses, “systems and methods for the automatic identification and tracking of objects in a variety of settings based on a physical shape, geometry, and appearance of the object”), wherein the method comprises the steps: a) acquiring an image (ibc) comprising a brightness and/or color information (bci) of a possible region of the container, b) acquiring a map (md) comprising a depth information (di) of the region (Carlo in [0085] discloses, “a depth camera may include one or more color cameras, which acquire the color information about an object, and one or more Infra-Red (IR) cameras which may be used in conjunction with an IR structured-light illuminator to capture geometry information about the object”), characterized by: wherein the image (ibc) and the map (md) are acquired in top view (Carlo in [0060] discloses, “FIG. 16 depicts an arrangement of a scanning system according to one embodiment of the present invention in which a scanner is mounted on an overhead frame that is configured to capture overhead images of a cart as it passes under the scanner”).
Carlo doesn’t disclose about the following limitation as further recited in the claim.
Seunghyeok discloses c) determining the container information (coi) by fusing the brightness and/or color information (bci) and the depth information (di) (Seunghyeok in [0076] discloses, “The RGB-D fusion backbone extracts a color feature and a depth feature and fuses the extracted color feature and depth feature to generate a color-depth fusion feature”),
wherein the brightness and/or color information (bci) and/or the depth information (di) are/is about at least one boundary (bou) between two spatial regions occupied by different matter (Seunghyeok in [0145] discloses, “The RGB-D fusion backbone 404 may use color information or depth information to derive the boundary of the object instance”. Furthermore, Seunghyeok in [0130] discloses about different matter and region, “a bounding box having a closed curve may be generated between a boundary between the foreground object instance and the background and a boundary between one object instance and the other object instance”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Seunghyeok into the system of Carlo because using both color or brightness and depth information would allow the system to determine the container information more accurately.
Carlo and Seunghyeok in the combination doesn’t disclose about the following limitation as further recited in the claim.
Volker discloses the container information (coi) is about an absence or a presence of the container in the region (Volker in [0019] discloses, “Determining whether the rack comprises at least one container”), a position (POS) of the container (Volker in [0114] discloses, “Each possible position for a tube is indicated with position markers, e.g. a 2d barcode), a presence or an absence of a cap on the container (Volker in [0087] discloses, “Cap or no-cap properties are determined”), an absence or a presence of a laboratory sample in the container (Volker in [0158] discloses, “For empty container detection, the images are processed to check whether the test container is empty or not”) and/or a level (LE) of the sample in the container, and wherein the method comprises the step: d) transporting, gripping, decapping, capping, filling and/defilling the container based on the determined container information (coi) (Claim 1 limitation is treated as disjunctive condition meaning is “either/or”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Volker into the system of Carlo in view of Seunghyeok because it would allow the system to better understand the actual tube situation to reduce the mistakes and allows the laboratory automation system to correctly transport or grip containers.
Summary of Citations (Carlo)
Paragraph [0005]; “systems and methods for the automatic identification and tracking of objects in a variety of settings based on a physical shape, geometry, and appearance of the object. Some aspects of embodiments of the present invention relate to the use of depth camera systems configured to capture depth maps of scenes and to automatically generate three-dimensional (3-D) models of scenes in order to automatically detect and identify objects within the scene and to track the locations of the objects within the scene”.
Paragraph [0060]; “FIG. 16 depicts an arrangement of a scanning system according to one embodiment of the present invention in which a scanner is mounted on an overhead frame that is configured to capture overhead images of a cart as it passes under the scanner”.
Paragraph [0085]; “a depth camera may include one or more color cameras, which acquire the color information about an object, and one or more Infra-Red (IR) cameras which may be used in conjunction with an IR structured-light illuminator to capture geometry information about the object”.
Summary of Citations (Seunghyeok)
Paragraph [0076]; “The RGB-D fusion backbone extracts a color feature and a depth feature and fuses the extracted color feature and depth feature to generate a color-depth fusion feature”.
Paragraph [0130]; “a bounding box having a closed curve may be generated between a boundary between the foreground object instance and the background and a boundary between one object instance and the other object instance”.
Paragraph [0145]; “The RGB-D fusion backbone 404 may use color information or depth information to derive the boundary of the object instance”.
Summary of Citations (Volker)
Paragraph [0019]; “Determining whether the rack comprises at least one container”.
Paragraph [0087]; “Cap or no-cap properties are determined”.
Paragraph [0114]; “Each possible position for a tube is indicated with position markers, e.g. a 2d barcode. A region of interest (ROI) is defined in an area where back illumination is detected and a tube has to be visible if present”.
Paragraph [0158]; “For empty container detection, the images are processed to check whether the test container is empty or not”.
Regarding claim 2, Carlo in the combination discloses the method according to claim 1, wherein step a) comprises: acquiring the image (ibc) comprising the brightness and/or color information (bci) by detecting visible light (vL), and/or wherein step b) comprises: acquiring the map (md) comprising the depth information (di) by detecting infrared light (iL) (Carlo in [0085] discloses, “a three-dimensional scanner includes one or more depth cameras, where a depth camera may include one or more color cameras, which acquire the color information about an object, and one or more Infra-Red (IR) cameras which may be used in conjunction with an IR structured-light illuminator to capture geometry information about the object”).
Summary of Citations (Carlo)
Paragraph [0085]; “a three-dimensional scanner includes one or more depth cameras, where a depth camera may include one or more color cameras, which acquire the color information about an object, and one or more Infra-Red (IR) cameras which may be used in conjunction with an IR structured-light illuminator to capture geometry information about the object”.
Regarding claim 4, Carlo in the combination discloses the method according to claim 1, wherein the image (ibc) and the map (md) are acquired by a 3D camera (Carlo in [0085] discloses, “a three-dimensional scanner includes one or more depth cameras, where a depth camera may include one or more color cameras, which acquire the color information about an object”).
Summary of Citations (Carlo)
Paragraph [0085]; “a three-dimensional scanner includes one or more depth cameras, where a depth camera may include one or more color cameras, which acquire the color information about an object, and one or more Infra-Red (IR) cameras which may be used in conjunction with an IR structured-light illuminator to capture geometry information about the object”.
Regarding claim 5, Seunghyeok in the combination discloses the method according to claim 1, wherein step c) comprises: extracting the brightness and/or color information (bci) from the image (ibc) and/or the depth information (di) from the map (md) in form of features (feat), and/or wherein step c) comprises: classifying the brightness and/or color information (bci) and/or the depth information (di) and fusing the classified brightness and/or color information (bci) and/or the classified depth information (di) (Seunghyeok in [0076] discloses, “The RGB-D fusion backbone extracts a color feature and a depth feature and fuses the extracted color feature and depth feature to generate a color-depth fusion feature. The color feature, the depth feature, and the color-depth fusion feature may be used as image features for detecting a foreground object instance from the cluttered scene image”. Furthermore, Seunghyeok in [0088] discloses about using the extracted color or depth features to determine a foreground or background class, “The bounding box dividing unit may be configured by two fully connected layers and the boundary box feature F.sub.B is supplied to one fully connected layer to extract a class and the boundary box feature F.sub.B is supplied to the other fully connected layer to extract a bounding box for an arbitrary foreground object instance”).
Summary of Citations (Seunghyeok)
Paragraph [0076]; “The RGB-D fusion backbone extracts a color feature and a depth feature and fuses the extracted color feature and depth feature to generate a color-depth fusion feature. The color feature, the depth feature, and the color-depth fusion feature may be used as image features for detecting a foreground object instance from the cluttered scene image”.
Paragraph [0088]; “The bounding box dividing unit may be configured by two fully connected layers and the boundary box feature F.sub.B is supplied to one fully connected layer to extract a class and the boundary box feature F.sub.B is supplied to the other fully connected layer to extract a bounding box for an arbitrary foreground object instance”.
Regarding claim 6, Seunghyeok in the combination discloses the method according to claim 1, wherein step c) is performed by an artificial neural network (AI), rule-based decisions and/or machine learning (Seunghyeok in [0136] discloses, “In the machine learning, the convolutional neural network (CNN) is one of an artificial neural network which is successfully applied to a field of visual image analysis”).
Summary of Citations (Seunghyeok)
Paragraph [0136]; “In the machine learning, the convolutional neural network (CNN) is one of an artificial neural network which is successfully applied to a field of visual image analysis”.
Regarding claim 7, apparatus claim 7 corresponds to method claim 1. Therefore, the
rejection analysis and motivation to combine of claim 1 is applicable to claim 7.
Claim 3 is rejected under 35 U.S.C 103 as being unpatentable over Carlo in view of Seunghyeok and Volker and further in view of Cheng US Patent Application Publication No. US-20190306489-A1 (hereinafter Cheng).
Regarding claim 3, Carlo in the combination discloses the method according to claim 1.
Carlo, Seunghyeok and Volker in the combination doesn’t disclose about the following limitation as further recited in the claim.
Cheng discloses step b) comprises: acquiring the map (md) comprising the depth information (di) by a time-of-flight-measurement (TOF) (Cheng in [0027] and Fig. 1 discloses about TOF, “In the various techniques for range sensing and depth estimation to achieve computer stereo vision, such as structured-light, active stereo, and time-of-flight (TOF)”).
It would have been obvious to one of ordinary skill in art before the effective filling date of the claimed invention to integrate the technique of Cheng into the system of Carlo in view of Seunghyeok and Volker because it would allow the system to acquire the depth information quickly with less computational resources.
Summary of Citations (Cheng)
Paragraph [0027]; “In the various techniques for range sensing and depth estimation to achieve computer stereo vision, such as structured-light, active stereo, and time-of-flight (TOF)”.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to ZAID MUHAMMAD SALEH whose telephone number is (703)756-1684.
The examiner can normally be reached M-F 8 am - 5 pm ET. Examiner interviews are available
via telephone, in-person, and video conferencing using a USPTO supplied web-based
collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO
Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.If attempts to
reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be
reached on (571)272-7332. The fax phone number for the organization where this application or
proceeding is assigned is 571-273-8300. Information regarding the status of published or
unpublished applications may be obtained from Patent Center. Unpublished application
information in Patent Center is available to registered users. To file and manage patent
submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center
and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For
additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative, call 800-786
9199 (IN USA OR CANADA) or 571-272-1000.
/ZAID MUHAMMAD SALEH/
Examiner, Art Unit 2668
06/16/2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights