DETAILED ACTION
Status of Claims
Claims 1-20 are currently pending and have been examined in this application. This Non-final communication is the first action on the merits.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 9/11/2024, 1/10/2025, 6/17/2025, 8/15/2025, and 10/15/2025 were filed in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Double Patenting
The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A non-statutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a non-statutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-8, 10, and 12-20 are rejected on the ground of non-statutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12,128,567. Although the claims at issue are not identical, they are not patentably distinct from each other because they are coextensive in scope to the allowed claims and would be fully encompassed and/or anticipated by the issued U.S. Patent.
Specifically wherein;
Regarding claim 1, Applicant provides similar limitations as in claim 7 of the issued U.S. Patent, wherein both of the respective claim(s) include (similar limitations provided in bold):
A system, comprising: one or more processors configured to: identify an object as a variant of an object type by inputting sensed data associated with the object into a
modified machine learning model corresponding to the variant of the object type, wherein the modified machine learning model corresponding to the variant of the object type is generated using a machine learning model corresponding to the object type; determine a sorting parameter associated with a sorting operation to be performed on the object based at least in part on the variant of the object type; and
provide a control signal to a sorting device to perform the sorting operation on the object in accordance with the sorting parameter; and a memory coupled to the one or more processors and configured to provide the one or more processors with instructions.
Although conflicting claims are not identical, they are not patentably distinct from each other because removing inherent and/or unnecessary limitation(s)/step(s) or adding an element and its function would be within the level of one of ordinary skill in the art. It is well settled that the adding or deleting of an element and its function(s) in the claim of the present application are an obvious expedient if the remaining elements perform the same function as before. In re Karlson, 136 USPQ 184 (CCPA 1963). Also note Ex parte Rainu, 168 USPQ 375 (Bd. App. 1969). Omission of a referenced element or step whose function is not needed would be obvious to one of ordinary skill in the art. Examiner further notes wherein although the claims are not identical (slightly broader), they are commensurate in scope to the claim limitations provided in the issued U.S. Patent, and likewise would anticipate the currently provided claim limitations.
The Examiner would like to note, that claims 6 and 7 of the co-pending application, which is a dependent claim of claim 1 of the co-pending application, also comprises all of the limitations of claim 1 of the co-pending application for its dependency.
Regarding claims 2-8, 10, 12-16 and 19-20 Applicant provides similar limitations as provided in at least claims 2-14 and 16-19 of the issued U.S. Patent. Although conflicting claims are not identical, they are not patentably distinct from each other because removing inherent and/or unnecessary limitation(s)/step(s) or adding an element and its function would be within the level of one of ordinary skill in the art. It is well settled that the adding or deleting of an element and its function(s) as in the claims of the present application are an obvious expedient if the remaining elements perform the same function as before. In re Karlson, 136 USPQ 184 (CCPA 1963). Also note Ex parte Rainu, 168 USPQ 375 (Bd. App. 1969). Omission of a referenced element or step whose function is not needed would be obvious to one of ordinary skill in the art. Examiner further notes wherein although the claims are not identical (slightly broader), they are commensurate in scope to the claim limitations provided in the issued U.S. Patent, and likewise would anticipate the currently provided claim limitations.
Regarding claim 17, Applicant provides similar limitations as in claim 17 of the issued U.S. Patent, wherein both of the respective claim(s) include (similar limitations provided in bold):
A method, comprising: identifying an object as a variant of an object type by inputting sensed data associated with the object into a modified machine learning model corresponding to the variant of the object type, wherein the modified machine learning model corresponding to the variant of the object type is generated using a machine learning model corresponding to the object type; determining a sorting parameter associated with a sorting operation to be performed on the object based at least in part on the variant of the object type; and providing a control signal to a sorting device to perform the sorting operation on the object in accordance with the sorting parameter.
Although conflicting claims are not identical, they are not patentably distinct from each other because removing inherent and/or unnecessary limitation(s)/step(s) or adding an element and its function would be within the level of one of ordinary skill in the art. It is well settled that the adding or deleting of an element and its function(s) in the claim of the present application are an obvious expedient if the remaining elements perform the same function as before. In re Karlson, 136 USPQ 184 (CCPA 1963). Also note Ex parte Rainu, 168 USPQ 375 (Bd. App. 1969). Omission of a referenced element or step whose function is not needed would be obvious to one of ordinary skill in the art. Examiner further notes wherein although the claims are not identical (slightly broader), they are commensurate in scope to the claim limitations provided in the issued U.S. Patent, and likewise would anticipate the currently provided claim limitations.
The Examiner would like to note, that claims 16 and 17 of the co-pending application, which is a dependent claim of claim 15 of the co-pending application, also comprises all of the limitations of claim 1 of the co-pending application for its dependency.
Examiner further notes wherein the non-statutory double patenting rejection(s) provided herein would be overcome with a timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 4, 9, and 17 are rejected under 35 U.S.C. 102 (a) (1) as being anticipated by Kumar (US 20180243800 A1)
Claim 1:
Kumar teaches the following limitations:
A system, comprising: one or more processors configured to: (Kumar - [0191] These program instructions may be provided to one or more processors and/or controller(s) of a general purpose computer, special purpose computer, or other programmable data processing apparatus (e.g., controller) to produce a machine, …) identify an object as a variant of an object type by inputting sensed data associated with the object into a modified machine learning model corresponding to the variant of the object type, (Kumar-[Abstract] A material sorting system sorts materials utilizing a vision system that implements a machine learning system in order to identify or classify each of the materials, which are then sorted into separate groups based on such an identification or classification. .; [0060] The vision system 110 captures visual images of each of the scrap pieces 101, for example, by using a typical optical sensor as utilized in typical digital cameras and video equipment.) wherein the modified machine learning model corresponding to the variant of the object type is generated using a machine learning model corresponding to the object type; (Kumar - [0052] … certain embodiments of the present disclosure can classify and sort into separate bins aluminum alloy scrap pieces classified as aluminum alloy 5086 separate from aluminum alloy scrap pieces classified as aluminum alloy 5022. Such an ability to sort scrap pieces of aluminum alloys from each other within a particular aluminum alloy series has never been accomplished before in the prior art.) ; [0091] … the machine learning algorithms extract features from the captured images using image processing techniques well known in the art. … machine learning algorithms learn the relationships between different types of materials and their features (e.g., as captured by the images, such as color, texture, hue, shape, brightness, etc.), creating a knowledge base for later classification of a heterogeneous mixture of scrap pieces received by the sorting system) determine a sorting parameter associated with a sorting operation to be performed on the object based at least in part on the variant of the object type; and provide a control signal to a sorting device to perform the sorting operation on the object in accordance with the sorting parameter; (Kumar - [0072] To accomplish this sort, when a scrap piece 101 is classified as falling into a predetermined grouping of classifications, the same sorting device may be activated to sort these into the same sorting bin. Such combination sorting may be applied to produce any desired combination of sorted scrap pieces. ; [126] Once the system and process 800 determines that a classified scrap piece is passing within the vicinity of a sorting device associated with that classification, it will activate that sorting device in the process block 805 in order to eject the classified scrap piece into the sorting bin associated with that classification) and a memory coupled to the one or more processors and configured to provide the one or more processors with instructions. (Kumar - [0200] One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to the local bus 3405 (e.g., through a PCI Bridge (not shown)). An integrated memory controller and cache memory may be coupled to the one or more processors 3415. The one or more processors 3415 may include one or more central processor units and/or one or more graphics processor units and/or one or more tensor processing units.)
Examiner Note:
The act of classification in Kumar corresponds to application of parameters in the instant application.
Alloy Series or Sub-Series corresponds to a Variant
Claim 4:
Kumar teaches the following limitations:
The system of claim 1, wherein the modified machine learning model is configured to recognize the variant of the object type.
(Kumar- [0052] … certain embodiments of the present disclosure can classify and sort into separate bins aluminum alloy scrap pieces classified as aluminum alloy 5086 separate from aluminum alloy scrap pieces classified as aluminum alloy 5022. Such an ability to sort scrap pieces of aluminum alloys from each other within a particular aluminum alloy series has never been accomplished before in the prior art.) ; [0091] … the machine learning algorithms extract features from the captured images using image processing techniques well known in the art. … machine learning algorithms learn the relationships between different types of materials and their features (e.g., as captured by the images, such as color, texture, hue, shape, brightness, etc.), creating a knowledge base for later classification of a heterogeneous mixture of scrap pieces received by the sorting system)
Claim 9:
Kumar teaches the following limitations:
The system of claim 1, wherein the sorting parameter associated with the sorting operation on the object comprises a specified location of a collection container in which to deposit the object.
(Kumar – [0052] … For example, certain embodiments of the present disclosure can classify and sort into separate bins aluminum alloy scrap pieces classified as aluminum alloy 5086 separate from aluminum alloy scrap pieces classified as aluminum alloy 5022. Such an ability to sort scrap pieces of aluminum alloys from each other within a particular aluminum alloy series has never been accomplished before in the prior art.)
Claim 17:
Kumar teaches the following limitations:
A method, comprising: identifying an object as a variant of an object type by inputting sensed data associated with the object into a modified machine learning model corresponding to the variant of the object type, (Kumar-[Abstract] A material sorting system sorts materials utilizing a vision system that implements a machine learning system in order to identify or classify each of the materials, which are then sorted into separate groups based on such an identification or classification. .; [0060] The vision system 110 captures visual images of each of the scrap pieces 101, for example, by using a typical optical sensor as utilized in typical digital cameras and video equipment.) wherein the modified machine learning model corresponding to the variant of the object type is generated using a machine learning model corresponding to the object type; (Kumar - [0052] … certain embodiments of the present disclosure can classify and sort into separate bins aluminum alloy scrap pieces classified as aluminum alloy 5086 separate from aluminum alloy scrap pieces classified as aluminum alloy 5022. Such an ability to sort scrap pieces of aluminum alloys from each other within a particular aluminum alloy series has never been accomplished before in the prior art.) ; [0091] … the machine learning algorithms extract features from the captured images using image processing techniques well known in the art. … machine learning algorithms learn the relationships between different types of materials and their features (e.g., as captured by the images, such as color, texture, hue, shape, brightness, etc.), creating a knowledge base for later classification of a heterogeneous mixture of scrap pieces received by the sorting system) determining a sorting parameter associated with a sorting operation to be performed on the object based at least in part on the variant of the object type; and providing a control signal to a sorting device to perform the sorting operation on the object in accordance with the sorting parameter. (Kumar - [0072] To accomplish this sort, when a scrap piece 101 is classified as falling into a predetermined grouping of classifications, the same sorting device may be activated to sort these into the same sorting bin. Such combination sorting may be applied to produce any desired combination of sorted scrap pieces. ; [126] Once the system and process 800 determines that a classified scrap piece is passing within the vicinity of a sorting device associated with that classification, it will activate that sorting device in the process block 805 in order to eject the classified scrap piece into the sorting bin associated with that classification)
Examiner Note:
The act of classification in Kumar corresponds to application of parameters in the instant application.
Alloy Series or Sub-Series corresponds to a Variant
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 3, 12, 13 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar (US 20180243800 A1) as modified by Dupree (US 20220288787 A1)
Claim 2:
Kumar does not explicitly teach the following limitations, however Dupree teaches:
The system of claim 1, wherein that the object cannot be identified from the sensed data associated with the object using the machine learning model corresponding to the object type is determined by a remote processor.
(Dupree - [0130] At 904, an attempt is made to match the item data to a model, e.g., a model comprising a library of item models. The library of item models may be dynamically updated such as via a machine learning process.; [0132] If it is determined at 906 that a match to an item-specific model cannot be found, at 910 the size, weight, shape, type of packaging, center of gravity, and/or other attributes of the item are determined and attempted to be matched to a model associated with items of that size, weight, shape, etc.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method of processing the variants identified by Kumar so that a newly discovered variant could be stored in a database of known objects and used to update the existing Machine Learning Model for future identification and evaluation of the target objects as taught in Dupree. This would provide a means of efficiently identifying future objects so that they could be quickly and accurately separated from unwanted materials thus increasing the purity of the recovered materials.
Claim 3:
Kumar does not explicitly teach the following limitations, however Dupree teaches:
The system of claim 2, wherein that the object cannot be identified from the sensed data associated with the object using the machine learning model corresponding to the object type is determined by the remote processor based on the machine learning model corresponding to the object type outputting a classification confidence corresponding to the object that is lower than a desired confidence threshold.
(Dupree- [0031] In various embodiments, 3D cameras, force sensors, and other sensors and/or sensor arrays are used to detect and determine attributes of items to be picked and/or placed. Items the type of which is determined (e.g., with sufficient confidence, as indicated by a programmatically determined confidence score, for example) may be grasped and placed using strategies derived from an item type-specific model.; [0130] At 904, an attempt is made to match the item data to a model, e.g., a model comprising a library of item models. The library of item models may be dynamically updated such as via a machine learning process.; [0132] If it is determined at 906 that a match to an item-specific model cannot be found, at 910 the size, weight, shape, type of packaging, center of gravity, and/or other attributes of the item are determined and attempted to be matched to a model associated with items of that size, weight, shape, etc.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method of processing the variants identified by Kumar so that a newly discovered variant could be given a confidence score, stored in a database of known objects and used to update the existing Machine Learning Model for future identification and evaluation of the target objects as taught in Dupree. This would provide a means of efficiently identifying future objects so that they could be quickly and accurately separated from unwanted materials thus increasing the purity of the recovered materials.
Claim 12:
Kumar teaches the following limitations:
The system of claim 1, wherein the one or more processors are further configured to: determine whether the sorting operation performed by the sorting device is successful; and
(Kumar- [0076] The conveyor system 103 may include a circular conveyor (not shown) so that unclassified scrap pieces are returned to the beginning of the sorting system 100 to be singulated by the singulator 106 and run through the system 100 again. Moreover, because the system 100 is able to specifically track each scrap piece 101 as it travels on the conveyor system 103, some sort of sorting device (e.g., the sorting device 129) may be implemented to eject a scrap piece 101 that the system 100 has failed to classify after a predetermined number of cycles through the sorting system 100 (or the scrap piece 101 is collected in bin 140).)
Kumar does not explicitly teach the following limitations, however Dupree teaches:
generate training data from a failed sorting operation.
(Dupree-[0130] At 904, an attempt is made to match the item data to a model, e.g., a model comprising a library of item models. The library of item models may be dynamically updated such as via a machine learning process.; [0132] If it is determined at 906 that a match to an item-specific model cannot be found, at 910 the size, weight, shape, type of packaging, center of gravity, and/or other attributes of the item are determined and attempted to be matched to a model associated with items of that size, weight, shape, etc. If at 912 it is determine that a match to such a model has been found, at 914 the determined model is used to determine a strategy to grasp, pick up, move, and/or place the item)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method of processing the variants identified by Kumar so that a newly discovered variant could be stored in a database of known objects and used to update the existing Machine Learning Model for future identification and evaluation of the target objects as taught in Dupree. This variant data along with the data generated from failed sorting operations would provide a means of efficiently identifying future objects so that they could be quickly and accurately separated from unwanted materials thus increasing the purity of the recovered materials.
Claim 13:
Kumar does not explicitly teach the following limitations, however Dupree teaches:
The system of claim 12, wherein the one or more processors are further configured to update the modified machine learning model corresponding to the variant of the object type based at least in part on the determination of the sorting operation performed by the sorting device is successful.(Dupree- [0033] In some embodiments, a robotic system as disclosed herein may engage in a process to gather and store (e.g., add to library) attributes and/or strategies to identify and pick/place an item of unknown or newly-discovered type. For example, the system may hold the item at various angles and/or locations to enable 3D cameras and/or other sensors to generate sensor data to augment and/or create a library entry that characterizes the item type and stores a model of how to identify and pick/place items of that type; [0130] At 904, an attempt is made to match the item data to a model, e.g., a model comprising a library of item models. The library of item models may be dynamically updated such as via a machine learning process.; [0132] … If at 912 it is determine that a match to such a model has been found, at 914 the determined model is used to determine a strategy to grasp, pick up, move, and/or place the item.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method of processing the variants identified by Kumar so that a newly discovered variant could be stored in a database of known objects and used to update the existing Machine Learning Model for future identification and evaluation of the target objects as taught in Dupree. This variant data along with the data generated from successful sorting operations would provide a means of efficiently identifying future objects so that they could be quickly and accurately separated from unwanted materials thus increasing the purity of the recovered materials.
Claim 15:
Kumar does not explicitly teach the following limitations, however Dupree teaches:
The system of claim 1, wherein the one or more processors are further configured to update a parameter within a data structure associated with the object to indicate that the object is associated with the variant of the object type.
(Dupree- [0133] At 918 it is determined whether processing the item has generated additional or new information about the item and/or item type. If so, at 920 the new information is used to generate and/or update a model for the item and/or item type. … ; [0134] In some embodiments, detection of a new item or item type triggers a discovery process in which attributes of the item may be determined and determined more fully or precisely. …)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method of processing the variants identified by Kumar so that a newly discovered variant could be stored in a database of known objects and used to update the existing Machine Learning Model for future identification and evaluation of the target objects as taught in Dupree. This would provide a means of efficiently identifying future objects so that they could be quickly and accurately separated from unwanted materials thus increasing the purity of the recovered materials.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Kumar (US 20180243800 A1) as modified by Dupree (US 20220288787 A1) In view of Robertson (US 20190261566 A1)
Claim 5:
Kumar does not explicitly teach the following limitations, however Dupree teaches:
The system of claim 1, wherein the one or more processors are further configured to generate training data used to train the modified machine learning model comprising to: determine previously recorded sensed data for which objects of the object type were classified with confidences lower than a desired confidence threshold using the machine learning model corresponding to the object type;
(Dupree - [0133] At 918 it is determined whether processing the item has generated additional or new information about the item and/or item type. If so, at 920 the new information is used to generate and/or update a model for the item and/or item type. New information may include a measured weight, a center of gravity, a type of packaging, etc. If no new information has been developed and/or once such information has been reflected in any applicable model(s), the process ends.; [0134] In some embodiments, detection of a new item or item type triggers a discovery process in which attributes of the item may be determined and determined more fully or precisely. For example, the system may use the robotic arm to pick up the item and hold it in different locations and/or at different orientations to generate (additional) images or views of the item to be used to model or more completely model the item.)
Kumar in combination with Dupree does not explicitly teach the following limitations, however Robertson teaches:
and receive annotations corresponding to the previously recorded sensed data.
(Robertson – [0111] … To provide training data, images obtained from representative viewpoints are annotated manually with the position and/or extent of target fruit. Various embodiments of this idea are possible: [0112] 1. A decision forest classifier or convolutional neural network (CNN) may be trained to perform semantic segmentation, i.e. to label pixels corresponding to ripe fruit, unripe fruit, and other objects.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method of processing the variants identified by Kumar so that a newly discovered variant could be stored in a database of known objects and used to update the existing Machine Learning Model for future identification and evaluation of the target objects as taught in Dupree. This would provide a means of efficiently identifying future objects so that they could be quickly and accurately separated from unwanted materials thus increasing the purity of the recovered materials.
Further, it would have been obvious to one of ordinary skill in the art to modify Kumar and Dupree to include a means for annotating sensor data as taught in Robertson. Having the ability to annotate incoming data provides for more accurate identification of target objects for the machine learning system as it refines the in the object sorting process.
Claim(s) 6, 8, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar (US 20180243800 A1) as modified by Rodrigues (US 10583560 B1)
Claim 6:
Kumar does not explicitly teach the following limitations, however Rodrigues teaches:
The system of claim 1, wherein the sorting parameter associated with the sorting operation on the object comprises a specified location on the object on which the sorting device is to contact or emit airflow with respect to the object.
(Rodrigues-[Page 17, Column 18, lines 28- 33] … it has been discovered that generating the object handling strategy 442 based on the object identity approximation 360...enables the robotic system 100 to optimize; the object handling strategy 442 to account for a wider range of the detectable object properties 352.; [Page 17, Column 18, lines 62- 67 ] In the case for the object entry properties 308 of the deformation mode 324, as an example, the object handling strategy 442 can include instructions on a grip position on the target object 112 to limit or minimize the amount of shape distortion, such as bending or sagging, of the target object 112.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar with Rodrigues to include a sorting parameter for the control signal that would provide guidance for the sorting device to more efficiently manipulate target objects by controlling the amount of force and the object contact location, thereby increasing the accuracy of the sorting operation.
Claim 8:
Kumar does not explicitly teach the following limitations, however Rodrigues teaches:
The system of claim 1, wherein the sorting parameter associated with the sorting operation on the object comprises a specified force with which the sorting device is to contact or emit airflow with respect to the object.
(Rodrigues - [Page 17, Column 18, lines 54- 62] … as an example, the object handling strategy 442 can include instructions to limit the amount of grip pressure that can be applied to the target object 112 by the gripping device 334 of the object handling unit 328. For example, the object handling strategy 442 can include limitations on the contact pressure applied by the end-effector 332 or the amount of force applied by the gripping device 334 of FIG. 3 when contacting the target object 112.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar with Rodrigues to include a sorting parameter for the control signal that would provide guidance for the sorting device to more efficiently manipulate target objects by controlling the amount of force and the object contact location, thereby increasing the accuracy of the sorting operation.
Claim 18:
Kumar does not explicitly teach the following limitations, however Rodrigues teaches:
The method of claim 17, wherein the sorting parameter associated with the sorting operation on the object comprises a specified location on the object on which the sorting device is to contact or emit airflow with respect to the object.
(Rodrigues-[Page 17, Column 18, lines 28- 33] … it has been discovered that generating the object handling strategy 442 based on the object identity approximation 360...enables the robotic system 100 to optimize; the object handling strategy 442 to account for a wider range of the detectable object properties 352.; [Page 17, Column 18, lines 62- 67 ] In the case for the object entry properties 308 of the deformation mode 324, as an example, the object handling strategy 442 can include instructions on a grip position on the target object 112 to limit or minimize the amount of shape distortion, such as bending or sagging, of the target object 112.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar with Rodrigues to include a sorting parameter for the control signal that would provide guidance for the sorting device to more efficiently manipulate target objects by controlling the amount of force and the object contact location, thereby increasing the accuracy of the sorting operation.
Claim 20:
Kumar does not explicitly teach the following limitations, however Rodrigues teaches:
The method of claim 17, wherein the sorting parameter associated with the sorting operation on the object comprises a specified force with which the sorting device is to contact or emit airflow with respect to the object.
(Rodrigues - [Page 17, Column 18, lines 54- 62] … as an example, the object handling strategy 442 can include instructions to limit the amount of grip pressure that can be applied to the target object 112 by the gripping device 334 of the object handling unit 328. For example, the object handling strategy 442 can include limitations on the contact pressure applied by the end-effector 332 or the amount of force applied by the gripping device 334 of FIG. 3 when contacting the target object 112.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar with Rodrigues to include a sorting parameter for the control signal that would provide guidance for the sorting device to more efficiently manipulate target objects by controlling the amount of force and the object contact location, thereby increasing the accuracy of the sorting operation.
Claims 7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar (US 20180243800 A1) as modified by Gorumkonda (US 10958895 B1)
Claim 7:
Kumar does not explicitly teach the following limitations, however Gorumkonda teaches:
The system of claim 1, wherein the sorting parameter associated with the sorting operation on the object comprises a specified angle from which the sorting device is to contact or emit airflow with respect to the object.
(Gorumkonda - [Page 18, Column 12, lines 55- 67] Additionally, the robotic arm controller 390 could determine an estimated pose of the particular item at the designated location, using the 3D model. … The robotic arm controller 390 could then use the estimated pose of the particular item to determine an optimal way to control the robotic picking arm 392, in order to best retrieve the particular item. For example, the robotic arm controller 390 could determine an angle of approach,…)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar with Gorumkonda to include a sorting parameter for the control signal that would provide guidance for the sorting device to more efficiently manipulate target objects by controlling the angle of approach, thereby increasing the accuracy of the sorting operation.
Claim 19:
Kumar does not explicitly teach the following limitations, however Gorumkonda teaches:
The method of claim 17, wherein the sorting parameter associated with the sorting operation on the object comprises a specified angle from which the sorting device is to contact or emit airflow with respect to the object.
(Gorumkonda - [Page 18, Column 12, lines 55- 67] Additionally, the robotic arm controller 390 could determine an estimated pose of the particular item at the designated location, using the 3D model. … The robotic arm controller 390 could then use the estimated pose of the particular item to determine an optimal way to control the robotic picking arm 392, in order to best retrieve the particular item. For example, the robotic arm controller 390 could determine an angle of approach,…)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar with Gorumkonda to include a sorting parameter for the control signal that would provide guidance for the sorting device to more efficiently manipulate target objects by controlling the angle of approach, thereby increasing the accuracy of the sorting operation.
Claim(s) 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar (US 20180243800 A1) as modified by Robertson (US 20190261566 A1)
Claim 10:
Kumar does not explicitly teach the following limitations, however Robertson teaches:
The system of claim 1, wherein the one or more processors are further configured to generate synthetic data for training the modified machine learning model corresponding to the variant of the object type.
(Robertson - [0120] An innovative approach to recovering the 3D shape of a candidate fruit from one or more images is to adapt the parameters of a generative model of the fruit's image appearance to maximize the agreement between the images and the model's predictions, e.g. by using Gauss-Newton optimization. This approach can also be used to refine a coarse initial estimate of the fruit's position and orientation (provided as described above). A suitable model could take the form of a (possibly textured) triangulated 3D mesh projected into some perspective views. The shape of the 3D mesh could be determined by a mathematical function of some parameters describing the shape of the fruit. A suitable function could be constructed by obtaining 3D models of a large number of fruits, and then using Principal Component Analysis (or other dimensionality reduction strategy) to discover a low-dimensional parameterization of the fruit's geometry. …)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method for introducing synthetic data to the machine learning model as taught in Robertson. Having the ability to enter synthetic data to the machine learning system provides for more control when teaching the robot to identify and select variations of similar target objects.
Claim 11:
Kumar does not explicitly teach the following limitations, however Robertson teaches:
The system of claim 10, wherein the one or more processors are further configured to generate the synthetic data by: obtaining a three-dimensional (3D) model of the object type; manipulating the 3D model of the object type to display a plurality of appearances of the 3D model of the object type; and generating a set of two-dimensional (2D) images of the plurality of appearances of the 3D model of the object type.
(Robertson - [0120] An innovative approach to recovering the 3D shape of a candidate fruit from one or more images is to adapt the parameters of a generative model of the fruit's image appearance to maximize the agreement between the images and the model's predictions, e.g. by using Gauss-Newton optimization. This approach can also be used to refine a coarse initial estimate of the fruit's position and orientation (provided as described above). A suitable model could take the form of a (possibly textured) triangulated 3D mesh projected into some perspective views. The shape of the 3D mesh could be determined by a mathematical function of some parameters describing the shape of the fruit. A suitable function could be constructed by obtaining 3D models of a large number of fruits, and then using Principal Component Analysis (or other dimensionality reduction strategy) to discover a low-dimensional parameterization of the fruit's geometry. …)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method for introducing 3D models to simulate variations of target objects for the machine learning model as taught in Robertson. Having the ability to introduce a variety of 3D models to the machine learning system provides for more control and variety when teaching the robot to identify and select variations of similar objects.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kumar (US 20180243800 A1) as modified by Dupree (US 20220288787 A1) in view of Wellman (US 20160167228 A1)
Claim 14:
Kumar in combination with Dupree does not explicitly teach the following limitations, however Wellman teaches:
The system of claim 12, wherein the one or more processors are configured to make the training data from the failed sorting operation available to a remote processor; and wherein the remote processor is configured to update the modified machine learning model corresponding to the variant of the object type at least in part based upon the training data from the failed sorting operation.
(Wellman – [0017] … For example, a target item, or characteristics thereof, may be identified, such as by optical or other sensors, in order to determine a grasping strategy for the item. The grasping strategy may be based at least in part upon a database containing information about the item, characteristics of the item, and/or similar items, such as information indicating grasping strategies that have been successful or unsuccessful for such items in the past. Entries or information in the database may be originated and/or updated based on human input for grasping strategies, determined characteristics of a particular item, and/or machine learning related to grasping attempts of other items sharing characteristics with the particular item. Embodiments herein include aspects directed to generating and/or accessing such databases.; [0072] … The grasping strategy evaluation module 740 can receive information about a grasping action performed in response to instructions provided by the grasping strategy instruction module 735 and evaluate a success of the grasping strategy for the item or attributes of the item involved in the particular grasping action. … )
Therefore, prior to the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to further modify Kumar and Dupree with Wellman in order to share the sensed data and modified Machine Learning Model associated with the variant objects across a broader network of processors thereby sharing the learned objects and updated Machine Learning Model with other machines and facilities, further increasing the efficiency of object recovery and the purity of recovered objects across a broader network.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Kumar (US 20180243800 A1) as modified by Wozniak (US 11922368 B1) In view of Robertson (US 20190261566 A1)
Claim 16:
Kumar teaches the following limitations:
The system of claim 1, wherein the one or more processors are further configured to generate the modified machine learning model corresponding to the variant of the object type by
(Kumar-[Abstract] A material sorting system sorts materials utilizing a vision system that implements a machine learning system in order to identify or classify each of the materials, which are then sorted into separate groups based on such an identification or classification. .; [0060] The vision system 110 captures visual images of each of the scrap pieces 101, for example, by using a typical optical sensor as utilized in typical digital cameras and video equipment.)
Kumar does not explicitly teach the following limitations, however Wozniak teaches:
adding a new output layer corresponding to the machine learning model corresponding to the object type
(Wozniak – [Page16, column 13, lines 30 – 37] In some embodiments, the central computer system 130 may also utilize the data (e.g., cluster data) that is output from the second machine learning model (e.g., via the unsupervised learning process), described above, to retrain a first machine learning model of one or more of the robotic systems. For example, the central computer system 130 may have previously trained the first ML model using first training data. …)
Kumar in combination with Wozniak does not explicitly teach the following limitations, however Robertson teaches:
by training the machine learning model using annotated sensed data pertaining to objects of the variant of the object type.
(Robertson – [0111] … To provide training data, images obtained from representative viewpoints are annotated manually with the position and/or extent of target fruit. Various embodiments of this idea are possible: [0112] 1. A decision forest classifier or convolutional neural network (CNN) may be trained to perform semantic segmentation, i.e. to label pixels corresponding to ripe fruit, unripe fruit, and other objects.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Kumar to include a method of providing output data that can be made available to various machine learning models as taught in Wozniak. Having the internal data produced by the machine learning model available for output provides a means for sharing data with other learning models and improving the existing machine learning model currently in use. Further, it would have been obvious to one of ordinary skill in the art to modify Kumar and Wozniak to include a means for annotating sensor data as taught in Robertson. Having the ability to annotate incoming data provides for more accurate identification of target objects for the machine learning system as it refines the in the object sorting process.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure or directed to the state of the art is listed on the enclosed PTO-892.
The following is a brief description for relevant prior art that was cited but not applied:
Wen (US 20210319545 A1) describes a vision inspection system includes a sorting platform having an upper surface supporting parts for inspection, wherein the parts are configured to be loaded onto the upper surface of the sorting platform in a random orientation. The vision inspection system includes an inspection station including an imaging device. The vision inspection system includes a vision inspection controller receiving images and processing the images based on an image analysis model to determine inspection results for each of the parts. The vision inspection controller has a shape recognition tool configured to recognize the parts in the field of view regardless of the orientation of the parts on the sorting platform. The vision inspection controller has an AI learning module operated to customize and configure the image analysis model based on the images received from the imaging device.
Sun (US 20210024297 A1) describes a system for sorting moving objects is disclosed. The system comprises a light source, an image capturing device, a controlling and processing device, and an object sorting device. Particularly, the controlling and processing device is configured to decide a first setting parameter so as to apply a parameter adjustment to the light source, and is also configured to decide a second setting parameter so as to apply a parameter adjustment to the image capturing device. After deciding an object classifier based on the first setting parameter, the second setting parameter, and object images received from the image capturing device, the object sorting device is controlled to apply an object sorting process to the of objects that are delivered by the belt conveyor, thereby sorting the objects into at least two object group consisting of a normal object group and a defective object group.
Valpola (US 20130266205 A1) describes an invention that relates to a method and system for recognizing physical objects. In the method an object is gripped with a gripper, which is attached to a robot arm or mounted separately. Using an image sensor, a plurality of source images of an area comprising the object is captured while the object is moved with the robot arm. In particular, the present invention relates to a method for the filtering of target object images in a robot system. Moving image elements are extracted from the plurality of source images by computing a variance image from the source images and forming a filtering image from the variance image. A result image is obtained by using the filtering image as a bitmask. The result image is used for classifying the gripped object.
Yu (US 20210023717 A1) describes an integrated robotic system and method that verifies an initial object estimation and then uses various devices and methods in refining or adjusting the object detection results. Using 2D or 3D image sensors, the system will verify or update an initial object estimation to increase the accuracy of an object detection for objects that are moving on a parcel conveyor.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN LINDSAY OSTROW whose telephone number is (703)756-1854. The examiner can normally be reached M-F 8 - 5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached on (571) 270 5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALAN LINDSAY OSTROW/Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657