Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The action is in response to the Applicant’s communication filed on 09/23/2024.
Claims 1-12 are pending, where claims 1 and 8 are independent.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/23/2024 has been filed on the filing date of the application. The submission is in-compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification objections (Title)
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: METHOD AND SYSTEM OF ANN BASED CRANE HANDALING FOR QUALITY ASSESSMENT OF OBJECTS IN AN INDUSTRIAL ENVIRONMENT. MPEP 606.01
Claim Objections
Claim 2 is objected to because of the following informalities:
Claim 2 recites the terminology "and/or", what is actually being performed by the alternatively claimed language. However, it will be assumed "or” for the purposes of examination.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-12 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Kristensen, USPGPub No. 20250178200 A1.
As to claim 1, Kristensen discloses A method for managing a crane system capable of handling an object, (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13) the method comprising:
generating an image data stream based on a plurality of images of the object captured by cameras the crane system; analyzing the image data stream by employing a computing unit of the crane system using an artificial neural network, wherein the artificial neural network is trained for identifying markers from the image data stream; determining, by the computing unit, object properties associated with the object based on the analysis of the image data stream, wherein the object properties comprise at least a quality of the object; (Kristensen [0007-69] “operating a palletizer - receiving image data, using at least one camera, of at least one item intended to be handled by the palletizer - analyzing, using a machine-learning model, the received image data - determining, using the machine-learning model - generating a control signal for instructing the palletizer to stop the handling process - efficiency of handling processes increased - image data be a video sequence, a video stream, - in real-time, and/or a sequence of still images - machine learning refer to algorithms and statistical models - to perform a specific task - inferred from an analysis of historical and/or training data - anomaly detection (i.e., outlier detection) used - machine-learning model be an artificial neural network - to achieve a desired output for a given input” [abstract] see Fig. 1-13, receiving image data, using plurality of cameras, sensors, palletizer handles item by analyzing and determining received image data for generating a control signal in real-time, anomaly detection (outlier detection) used to achieve desired output for given input using machine-learning model an artificial neural network obviously provides the limitations) and
automatically operating the crane system for handling the object based on the object properties (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13).
Application and the reference Kristensen are analogous arts from the same field of endeavor and contain overlapping structural and functional similarities and both contain controlling the operation of crane system.
It would be therefore obvious to one having ordinary skill in the art at the time of the invention that controlling palletizer in handling of items are assumed as managing crane handling an object].
As to the independent claims 7-8, the claims recite similar limitations as the independent claim 1 and rejected using same rational as stated above.
As to claim 2, Kristensen further discloses The method according to claim 1, wherein generating the image data stream comprises performing: preprocessing each of the images captured by the cameras, wherein preprocessing comprises one or more of reducing noise in the images and enhancing contrast of the images; determining from the images a foreground associated with the object; and/or annotating the foreground from the images (Kristensen [0007-69] “receiving image data, using at least one camera, of at least one item intended to be handled by the palletizer - efficiency of handling processes increased - image data be a video sequence, a video stream, - in real-time, and/or a sequence of still images - machine learning refer to algorithms and statistical models - to perform a specific task - inferred from an analysis of historical and/or training data - content of images analyzed - machine-learning model to analyze the content of an image - training a machine-learning model using training sensor data and a desired output, the machine-learning model “learns” a transformation between the sensor data and the output - non-training sensor data provided to the machine-learning model - data (e.g., sensor data, meta data and/or image data) preprocessed to obtain a feature vector - used as input to the machine-learning model - anomaly detection (i.e., outlier detection) used - providing an identification of input values that raise suspicions by differing significantly from the majority of input or training data - trained using anomaly detection, and/or the machine-learning algorithm - anomaly detection component - machine-learning model be an artificial neural network - to achieve a desired output for a given input” [abstract] see Fig. 1-13, analyze content of image, preprocessed data (sensor data, meta data and/or image data) to obtain a feature vector used as input to the machine-learning model, anomaly detection (i.e., outlier detection), providing identified of input values that raise suspicions by differing significantly from the majority of input or training data for machine-learning algorithm to achieve a desired output for given input obviously provides preprocessing images captured by the cameras - reducing noise in the images and enhancing contrast of the images; determining from the images a foreground associated with the object; and/or annotating the foreground from the images).
As to claim 3, Kristensen further discloses The method according to claim 1, wherein analyzing the image data stream using the artificial neural network comprises detecting, from the image data stream, presence of one or more abnormalities associated with the object, and wherein the abnormalities comprise one or more of a defect in the object and a human in proximity of the object (Kristensen [0007-69] “operating a palletizer - receiving image data, using at least one camera, of at least one item intended to be handled by the palletizer - analyzing, using a machine-learning model, the received image data - determining, using the machine-learning model - generating a control signal for instructing the palletizer to stop the handling process - efficiency of handling processes increased - image data be a video sequence, a video stream, - in real-time, and/or a sequence of still images - machine learning refer to algorithms and statistical models - to perform a specific task - inferred from an analysis of historical and/or training data - anomaly detection (i.e., outlier detection) used - machine-learning model be an artificial neural network - to achieve a desired output for a given input” [abstract] see Fig. 1-13, receiving image data, using plurality of cameras, sensors, palletizer handles item by analyzing and determining received image data for generating a control signal in real-time, anomaly detection (outlier detection) used to achieve desired output for given input using machine-learning model an artificial neural network obviously provides analyzing the image data stream using the artificial neural network comprises detecting, from the image data stream, presence of one or more abnormalities associated with the object).
As to claim 4, Kristensen further discloses The method according to any one of the claim 1, wherein analyzing the image data stream using the artificial neural network comprises: segmenting the images based on the artificial neural network; identifying a distance between markers in the images; and determining, based on the distance and the segmented images, presence of the abnormalities (Kristensen [0007-69] “operating a palletizer - receiving image data, using at least one camera, of at least one item intended to be handled by the palletizer - analyzing, using a machine-learning model, the received image data - determining, using the machine-learning model - generating a control signal for instructing the palletizer to stop the handling process - image data be a video sequence, a video stream, - in real-time, and/or a sequence of still images - machine-learning model to analyze the content of an image - “learns” to recognize the content of the images - anomaly detection (i.e., outlier detection) used - providing an identification of input values that raise suspicions by differing significantly from the majority of input or training data - machine-learning model be an artificial neural network - to achieve a desired output for a given input” [abstract] see Fig. 1-13, receiving image data, using plurality of cameras, sensors, palletizer handles item by analyzing and determining received image data, machine-learning model to analyze the content of an image with anomaly detection providing identification (as marking) of input values that raise suspicions by differing significantly from the majority of input to achieve desired output obviously provides analyzing the image data stream using the artificial neural network comprises: segmenting the images based on the artificial neural network; identifying a distance between markers in the images; and determining, based on the distance and the segmented images, presence of the abnormalities).
As to claim 5, Kristensen further discloses The method according to claim 1, wherein automatically operating the crane system comprises operating a hoist of the crane system for handling the object based on predefined handling parameters defined based on the object properties during training of the artificial neural network (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13, automatically profiling palletizer (robots, portal cranes or hoists) with cameras identifying size and pallet content using machine-learning model (an artificial neural network) obviously provides automatically operating the crane system comprises operating a hoist of the crane system for handling the object based on predefined handling parameters defined based on the object properties during training of the artificial neural network).
As to claim 6, Kristensen further discloses The method according to claim 5, further comprising controlling at least one working parameter of one of the hoist and the crane system depending on positions of markers in the images of the image data stream (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13, plurality of cameras mounted with palletizer (robots, portal cranes or hoists) for generating control signal for instructing the palletizer identifying size and pallet content using machine-learning model (an artificial neural network) obviously provides controlling at least one working parameter of one of the hoist and the crane system depending on positions of markers in the images of the image data stream).
As to claim 9, Kristensen further discloses The crane system according to claim 8, wherein the control unit is configured to move the cameras, for capturing of the images of the object, along an axis of gantry tracks (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13, cameras mounted with palletizer (robots, portal cranes or hoists) for identifying size and pallet content using machine-learning model (an artificial neural network) obviously provides moving cameras, for capturing of the images of the object, along an axis of gantry tracks).
As to claim 10, Kristensen further discloses The crane system according to claim 8, comprising a first camera arranged at a first end of a gantry of the crane system, a second camera arranged at a second end of the gantry, and a third camera arranged in proximity of the hoist on the gantry (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13, plurality of cameras mounted with palletizer (robots, portal cranes or hoists) for generating control signal for instructing the palletizer identifying size and pallet content using machine-learning model (an artificial neural network) obviously provides the limitations).
As to claim 11, Kristensen further discloses A computing unit having an artificial neural network for managing a crane system, according to claim 8 (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13, controller of palletizer (robots, portal cranes or hoists) for generating control signal for instructing the palletizer identifying size and pallet content using machine-learning model (an artificial neural network) obviously provides the limitation).
As to claim 12, Kristensen further discloses A computer program product comprising a computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement the method according to claim 1 (Kristensen [0001-69] “controlling of a palletizer, e.g. a layer picker, based on machine-learning - palletizers used worldwide in the handling of items - in combination with robots, portal cranes or hoists - automatically profiling pallets by use of cameras for identifying size and content of the pallet” [abstract] see Fig. 1-13).
Citation of Pertinent Prior Art
It is noted that any citations to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the reference should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. See MPEP 2141.02 VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, i.e., as a whole and 2123.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The prior art made of record:
Dobler, et al. USPGPub No. 2020/0391981 A1 discloses an automated loading of a load by a crane system includes camera system to generate image data stream and analyzed by a computer unit with an artificial neural network.
Ivens, et al. USPGPub No. 20220084186 A1 discloses a shipping or cargo containers system for automatically inspecting shipping containers and assessing their condition using machine vision.
Pandya, et al. USP No. 12,411,212 B1 discloses a mobile system for managing safety in a dynamic environment and generating computer vision output data to move in the environment.
Penel, et al. USPGPub No. 2020/0307965 A1 discloses a mechanical cranes safety systems for cranes involve the cranes moving around loads that may weigh many tons.
Burkhardt, et al. USP No. 12,448,255 B1 discloses a crane hoist rope connected to a load handling and crane controller for controlling hoist rope for crane movements on the basis of imaging sensor of load handling on basis of model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Md Azad whose telephone @(571)272-0553 or email: md.azad@uspto.gov. The examiner can normally be reached on Mon-Thu 9AM-5PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mohammad Ali can be reached on (571)272-4105. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/Md Azad/
Primary Examiner, Art Unit 2119
Prosecution Insights