DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This action is in response to the Application filed on 12/20/2024.
Claims 1-18 are pending.
Priority
The present Application claims priority to KR-10-2023-0188834, with priority date 12/21/2023, which is acknowledged.
Information Disclosure Statement
The Information Disclosure Statement (IDS) filed on 12/20/2024 has been fully considered by the examiner.
Double Patenting
The examiner notes copending Application 18/991,100. In the non-final rejection of that case mailed on 07/29/2026, the reference claim 1 is rejected in view of the present claim 5 under provisional nonstatutory double-patenting grounds.
The examiner notes that the present claim set is interpreted to be narrower than those of the reference application, at least because of the claimed unloading process and layer information limitations, and thus no double patenting rejections of the present application are made.
Claim Rejections – 35 U.S.C. § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 are eligible for integrating abstract ideas into a practical application.
Analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added):
A method of providing classification information for iron scraps according to an unloading process, the method comprising:
receiving, by a receiving circuit, a loaded state image from a camera during an unloading process of a plurality of iron scraps loaded onto a loading device;
obtaining, by a processor, layer information that is updated as the unloading process progresses and determined based on a height of the plurality of iron scraps, wherein the height is determined in a stack direction of the plurality of iron scraps loaded onto the loading device;
determining, by the processor, a region of interest based on the loaded state image that is updated as the unloading process progresses;
generating, by the processor, a segmented image of a target iron scrap, which is included in the region of interest, wherein the target iron scrap is one of the plurality of iron scraps;
generating, by the processor, item information and grade information for the target iron scrap based on the segmented image and the layer information;
and providing, by the processor, classification information for the iron scraps, including the item information and the grade information.
Step 1: Does the claim belong to one of the statutory categories? Claim 1 is directed to a process, which is a statutory category of invention (YES).
Step 2A Prong One: Does the claim recite a judicial exception? Steps c, d, and f are regarded as mental processes that can be practically performed in the human mind. Step c requires obtaining layer information of the scraps based on a height in a stack direction, which is mentally performable via observation; note that step c does not require obtaining the layer information from the image. Step d requires determining a ROI based on the image as the unloading progresses, which is still mentally performable via any decision of an ROI in the image. Finally, step f recites generating item and grade information based on the segmented image and layer information; a mental determination would still read on this, as long as it takes the segmentation and layer information into account. Note that, although the claim requires these steps to be “by a processor”, the courts do not distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer (see MPEP 2106.04(a)(2).III) (YES).
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application? Steps a and b limit the claim to the particular field of iron scrap unloading from loading devices. Step e recites generating a segmented image of target scraps; although humans can mentally determine segmentations, they cannot mentally generate a segmented image as claimed. Finally, step g recites outputting the classification information. The claim integrates the mental processes into a practical application by tying them to the field of iron scrap unloading, as well as requiring the segmented image to be made from the ROI and the final classification information to be based on the layer information (YES). Claim 1 is eligible.
Similar analysis is applicable to independent claims 11 and 15, each of which additionally recite computerized systems at a high level of generality. Claim 11 is eligible. The examiner notes, however, that claim 15 and its dependents are rejected under 101 for different reasons at step 1 below.
Claims 2, 5-8, and 12 recite additional elements with no new judicial exceptions. Claims 2, 5-8, and 12 are eligible.
Claims 3-4 and 13-14 recite details of determining the ROI that are still mentally performable, but claims 1/11 still integrate these into a practical application. Claims 3-4 and 13-14 are eligible.
Claim 9 recites additional details of determining the ROI based on characteristics of a grapple, which also integrates the ROI determination into a practical application. Claim 9 is eligible.
Claim 10 recites determining layer change time points to obtain the layer information, based on critical heights and/or percentages, which can be performed mentally. However, claim 1 still integrates this into a practical application. Claim 10 is eligible.
Claims 16, 17, and 18 have similar analysis to claims 2, 3, and 4, respectively, but claims 16-18 are still rejected for different reasons at step 1 below.
Claims 15-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Regarding claim 15, the broadest reasonable interpretation of the term “computer-readable recording medium” as recited includes both transitory and non-transitory memories. A transitory memory or signal, while physical and real, does not possess concrete structure that would qualify as a device or part under the definition of a machine, is not a tangible article or commodity under the definition of a manufacture (even though it is man-made and physical in that it exists in the real world and has tangible causes and effects), and is not composed of matter such that it would qualify as a composition of matter, and thus does not fall within a statutory category (see MPEP 2106.03).
The examiner notes that paragraphs 0113-0114 of the originally-filed specification do state that the claimed storage/recording medium may be a non-transitory memory, but the claims are not limited to this embodiment by any limiting definition.
The examiner suggests amending line 1 of claim 15 to read “A non-transitory computer-readable recording medium” (emphasis added) to overcome this rejection.
Regarding claims 16-18, these claims are rejected for the same reasons, as they depend on claim 15 above. The examiner also requires amending the first line of each of these claims to include “non-transitory” if claim 15 is amended as suggested above.
Claim Rejections – 35 U.S.C. § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5-6, 11, and 15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chen et al. (Chinese Publ. CN-113642539-A).
The following cited paragraph numbers of Chen refer to the attached machine translation.
Regarding claim 1, Chen discloses a method of providing classification information for iron scraps according to an unloading process (see paragraphs 0037-0039, where the method is directed to scrap steel classification during a cargo unloading process), the method comprising:
receiving, by a receiving circuit, a loaded state image from a camera during an unloading process of a plurality of iron scraps loaded onto a loading device (see paragraphs 0023, 0042, where cargo images can be captured using drones or other aerial photography equipment; paragraphs 0039-0040, 0044 specify that images can be taken over time during an unloading process, each depicting a single layer of the scrap steel; paragraph 0039 states that the cargo loading device can be vehicles such as a truck or van);
obtaining, by a processor, layer information that is updated as the unloading process progresses and determined based on a height of the plurality of iron scraps, wherein the height is determined in a stack direction of the plurality of iron scraps loaded onto the loading device (paragraphs 0039-0040, 0107 specify that images are taken of each single surface layer of the cargo as the cargo height decreases, as a result of stacks being removed during unloading; paragraphs 0053-0055, 0058, 0066 specify that the information on each single layer corresponding to different heights is remembered for overall cargo grading);
determining, by the processor, a region of interest based on the loaded state image that is updated as the unloading process progresses (see paragraphs 0100-0103, 0141, where each new image can be pre-processed to identify the next single layer area revealed by the unloading and cropped for any key regions);
generating, by the processor, a segmented image of a target iron scrap, which is included in the region of interest, wherein the target iron scrap is one of the plurality of iron scraps (paragraphs 0051-0052 specify that various AI models can be used to generate instance segmentations of any of the scrap steel; paragraph 0084 specifies that different sub-regions of cargo can be identified using the mask-region network);
generating, by the processor, item information and grade information for the target iron scrap based on the segmented image and the layer information (see paragraphs 0047-0050, where quality grades of each cargo layer can be determined, such as for each of the above sub-regions for each layer; see paragraph 0055, where different types of cargo, such as impurities or foreign objects, can be recognized; object types such as steel or non-steel read as item information);
and providing, by the processor, classification information for the iron scraps, including the item information and the grade information (see paragraphs 0053-0055, 0058, 0066, where the overall item and grade info for the entire set of steel scrap can be obtained by combining the information from each single layer).
Regarding claim 5, Chen discloses the generating of the segmented image by the processor includes generating the segmented image that includes the target iron scrap from the loaded state image using a segmentation model (paragraphs 0051-0052 specify that various AI models can be used to generate instance segmentations of any of the scrap steel);
and the generating of the item information and the grade information by the processor includes generating the item information and the grade information for the target iron scrap using a classification model that performs analysis in units of images (see paragraphs 0052, 0059, where different classification models can be used, such as ResNet, DenseNet, or an image search approach).
Regarding claim 6, Chen discloses wherein the generating of the segmented image includes: performing, by the processor, instance segmentation on the loaded state image to generate a segmented image of a single object (paragraphs 0051-0052 specify that various AI models can be used to generate instance segmentations of any of the scrap steel);
and performing, by the processor, semantic segmentation on the loaded state image to generate a segmented image of aggregate objects (paragraph 0084 specifies that different sub-regions of cargo can be identified using the mask-region network; the examiner interprets these final sub-regions of multiple scraps as semantic segmentations as disclosed).
Regarding claim 11, Chen discloses an iron scrap classification apparatus for providing classification information for iron scraps according to an unloading process (see paragraph 0160 and system 500), the apparatus comprising:
a receiving circuit configured to receive a loaded state image from a camera during an unloading process of a plurality of iron scraps loaded onto a loading device (see paragraphs 0023, 0042, where cargo images can be captured using drones or other aerial photography equipment; see paragraph 0169, input/output devices 510, network interface 512);
and a processor that is configured to (see paragraph 0160, processors 502).
The remainder of claim 11 recites steps identical to those of claim 1. Therefore, Chen anticipates claim 11 as applied to claim 1 above.
Regarding claim 15, Chen discloses a computer-readable recording medium on which a program is recorded, the program comprising instructions that, when executed by a processor, cause the processor to (see paragraphs 0173-0175).
The remainder of claim 15 recites steps identical to those of claim 1. Therefore, Chen anticipates claim 15 as applied to claim 1 above.
Claim Rejections – 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 8, 10, 12, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (Chinese Publ. CN-113642539-A) in view of Viale (U.S. Publ. US-2023/0288142-A1).
Regarding claim 2, Chen fails to disclose the new limitations of claim 2. More specifically, Chen is silent as to the types of cameras used in the drones.
Pertaining to the same field of endeavor, Viale discloses wherein a method of obtaining the layer information includes a mono type method in which one camera is used or a stereo type method in which two or more cameras are used, and in the mono type method, depth information obtained through analysis of changes in wall surface of a loading box obtained from an image obtained by the one camera is used (see figures 1, 3, and paragraphs 0031-0032, where a sensor module mounted above buckets of metallic scrap material can use the stereo method option to create depth images of the scrap; as the camera is mounted above the buckets, depth is equivalent to height in this case).
Chen and Viale are considered analogous art, as they are both directed to image analysis of scrap metal layers. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Viale into Chen by using stereo cameras in Chen’s drones because doing so allows for measuring the height and volume of each scrap layer (see Viale paragraph 0032).
Regarding claim 8, Chen fails to disclose the new limitations of claim 8.
Pertaining to the same field of endeavor, Viale discloses wherein, in the stereo type method, the depth information obtained through analysis of changes in angles obtained from images for a same region that are obtained from the two or more cameras positioned on a same plane is used (see paragraphs 0031-0032, where stereo methods are known in the art to compare angles of features obtained from two different viewpoints/cameras; figure 6, monochrome cameras 112 and paragraph 0035 specify that the cameras can be aligned on a same plane of the device).
Chen and Viale are considered analogous art, as they are both directed to image analysis of scrap metal layers. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Viale into Chen by using stereo cameras in Chen’s drones because doing so allows for measuring the height and volume of each scrap layer (see Viale paragraph 0032).
Regarding claim 10, Chen fails to disclose the new limitations of claim 10.
Pertaining to the same field of endeavor, Viale discloses wherein the method of obtaining the layer information includes: obtaining, by the processor, a first layer change time point at which the height of the plurality of iron scraps are changed to a critical height or more based on the depth information obtained using the stereo type method; obtaining, by the processor, a second layer change time point at which a volume of the plurality of iron scraps are changed to a critical percentage or more based on an average volume of the loading device (see figure 4 and paragraphs 0029-0030, 0033, 0044, where the scrap material can be analyzed over time to determine if it has an orientation extending outside a predetermined working zone, which reads on exceeding a critical height, and to determine if it has a volume above a size threshold, which reads on exceeding a critical percentage);
and obtaining, by the processor, the layer information at at least one of the first layer change time point and the second layer change time point (paragraph 0033 specifies that the depth image / layer information is obtained in either case of the critical height or volume).
Chen and Viale are considered analogous art, as they are both directed to image analysis of scrap metal layers. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Viale into Chen by incorporating critical height/volume checks into Chen’s method because doing so sounds alarms if any scrap metal is too large or high (see Viale paragraphs 0029-0030, 0033, 0044).
Regarding claim 12, Chen in view of Viale discloses claim 12 as applied to claim 2 above.
Regarding claim 16, Chen in view of Viale discloses claim 16 as applied to claim 2 above.
Claims 3-4, 9, 13-14, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (Chinese Publ. CN-113642539-A) in view of Levine et al. ("Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection", Journal of Robotics Research paper, 2018).
Regarding claim 3, Chen fails to disclose the new limitations of claim 3.
Pertaining to the same field of endeavor, Levine discloses wherein the determining of the region of interest includes determining, by the processor, the region of interest based on a result of comparing a first loaded state image corresponding to a first time point with a second loaded state image corresponding to a second time point that is temporally later than the first time point (first see figure 4 and section 4.1, where a sequence of images
i
t
i
are captured of a robotic arm/grapple lifting generic materials; then see page 429, left column, where image subtraction can be used between images of different time points of arm operation).
Chen and Levine are considered analogous art, as they are both directed to image analysis of unloaded objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Levine into Chen by using image subtraction in Chen’s system because doing so indicates what objects were picked up and where, which functions as a ROI of new objects to examine (see Levine page 429, left column).
Regarding claim 4, Chen fails to disclose the new limitations of claim 4.
Pertaining to the same field of endeavor, Levine discloses wherein the determining of the region of interest includes determining, by the processor, the region of interest based on a difference region between the first loaded state image and the second loaded state image and an operation region of a grapple used in the unloading process (see citations to claim 3 above; the image subtraction creates a difference region; this region corresponds to the operation region of the arm/grapple).
Chen and Levine are considered analogous art, as they are both directed to image analysis of unloaded objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Levine into Chen by using image subtraction in Chen’s system because doing so indicates what objects were picked up and where, which functions as a ROI of new objects to examine (see Levine page 429, left column).
Regarding claim 9, Chen fails to disclose the new limitations of claim 9.
Pertaining to the same field of endeavor, Levine discloses the second time point is determined based on whether the grapple is included in any partial region of regions of a vertical direction of the loading device after the first time point (see page 429, left column, where the images to be subtracted require the arm to be visible above the bin);
and the determining of the region of interest includes, when the grapple is included in the partial region for a preset period of time or more, determining, by the processor, the region of interest based on an operating state of the grapple that indicates whether the grapple includes at least one iron scrap (see page 429, left column, where one of the images can be of the arm/grapple holding an object/iron scrap).
Chen and Levine are considered analogous art, as they are both directed to image analysis of unloaded objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Levine into Chen by using image subtraction in Chen’s system because doing so indicates what objects were picked up and where, which functions as a ROI of new objects to examine (see Levine page 429, left column).
Regarding claim 13, Chen in view of Levine discloses claim 13 as applied to claim 3 above.
Regarding claim 14, Chen in view of Levine discloses claim 14 as applied to claim 4 above.
Regarding claim 17, Chen in view of Levine discloses claim 17 as applied to claim 3 above.
Regarding claim 18, Chen in view of Levine discloses claim 18 as applied to claim 4 above.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (Chinese Publ. CN-113642539-A) in view of Kirillov et al. ("Panoptic Segmentation", IEEE/CVF CVPR paper, 10 April 2019).
Regarding claim 7, Chen fails to disclose the new limitations of claim 7.
Pertaining to the same field of endeavor, Kirillov discloses wherein the generating of the segmented image of the aggregate objects is performed on regions of the loaded state image, from which a region corresponding to the single object is excluded (first see section 3, where the panoptic segmentation model uses both semantic and instance segmentations; semantics are referred to as "stuff", whereas instances are referred to as "things"; then see page 8, right column, where the panoptic method starts with instance segmentations / single objects, then resolves the surrounding semantic segmentations / aggregate objects separately).
Chen and Kirillov are considered analogous art, as they are both directed to image analysis and segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Kirillov into Chen by using panoptic segmentation in Chen’s system because doing so would improve segmentations of individual scraps/"things" and the surrounding scraps/"stuff" (see Kirillov section 1, “Introduction”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS JOHN HELCO whose telephone number is (703)756-5539. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella, can be reached at telephone number 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/NICHOLAS JOHN HELCO/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights