CTNF 19/137,690 CTNF 97070 DETAILED ACTION 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Specification 06-31 AIA The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification. 06-11 AIA The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. 06-11-01 AIA The following title is suggested: Robotic Grasping and Re-grasping in Pick-and-Place Operations Or a similar title describing the specific nature of the robotic system claimed or to be claimed, rather than the present title which effectively describes any and all robotic systems . 07-30-03-h AIA Claim Interpretation The adjectives “first”, “second”, “third”, etc. have been interpreted merely to indicate that two items might, but are not required to be, different. It is common claim construction practice to use these adjectives for ease of referring to items while maintaining a potential distinction which might be further claimed in further limitations and dependent claims. The adjectives, under the broadest reasonable interpretation do not require a particular chronological order, difference, etc. except where Applicant’s specification sets for such an explicit definition or the claim otherwise specifies as such. Claim 3 recites the phrase “labeled data”. [0011] of Applicant’s disclosure states that “FIG. 3 is a diagram showing an example of labeled data”. However, it is unclear if this meets the normal meaning of the term, or is broader, as at most it appears to be grouping or pairing of (or as phrased later in the disclosure, combining (see e.g. [0021])) data together, whereas it is Examiner’s understanding that labelled data refers precisely to that, i.e. a label that is associated with a particular piece of data. Claim 3 even indicates that all of the data is part of the labeled data, rather than any particular data being the label itself. Furthermore, a label is not merely a numerical designation for a dataset. Thus, at present, Examiner is interpreting any combination of data used together as labeled data, in addition to its typical use of data having an assigned label which also appears at least hinted at in [0022] of Applicant’s specification “output data (a label and action content in this example)”. Claim Rejections - 35 USC § 112(b) 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claims 3 – 5 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding Claim 3, the claim recites “a trained neural network whose parameters are decided based on labeled data in which the first information, the second information, the third information, and the fourth information are designated as inputs and the action plan is designated as an output”. The statutory category of the claim is that of a machine, or apparatus (MPEP 2106) relates. A machine, or apparatus, may be claimed using structural limitations or as functional limitations thereof (MPEP 2114). The present claim language appears to merely be descriptive of the manner in which the neural network is operated. Furthermore, the claim completely omits the subject which actually performs the deciding or designating. In the interest of compact prosecution, the limitation has been interpreted as instead reading: a trained neural network; wherein the neural network comprises parameters which were trained using labeled data comprising the first information, the second information, the third information, and the fourth information as inputs and the action plan as an output. Examiner notes that Applicant’s disclosure appears to vary in whether the parameters are already trained and thus the model is already “trained”, training the model in the first place, and possibly of updating the model. As these are three wholly different approaches, the present limitation is directed only to the one believed attempted to be claimed through the use of “trained” in “trained neural network”. Regarding Claim 4, the claim recites the limitation “wherein the processor is configured to execute the instructions to … and generates the action plan including an action for changing the state of the target object in the area in a case where it is determined to change the state of the target object”. The limitation following “and” does not appear constructed grammatically appropriate, and furthermore does not appear to be a continuation of the preceding “determine …” limitation. Furthermore, the phrase “in a case where it is determined” is unclear. Finally, “the action plan including …” appears to lack sufficient antecedent basis. In the interest of compact prosecution, the limitation is instead interpreted as reading the following or equivalent: wherein the processor is configured to execute the instructions to … ; and if the determination of whether or not to change a state of the target object is to change a state of the target object, generating the action plan further comprises the action plan including an action for changing the state of the target object in the area. Regarding Claim 5, the claim recites the limitation “the robot configured to be controlled based on the action plan generated by the processing device”. There is insufficient antecedent basis for this limitation in the claim. It should instead read: “the robot, wherein the robot is configured to operate based on the action plan generated by the processing device”. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1 and 3 – 4, and 6 – 7 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 1 will be used to illustrate the rejection with respect to all independent claims (Claims 6 and 7), which are consequently rejected under the same logic presented below. Claim 1 recites: A processing device comprising: a memory configured to store instructions; and a processor configured to execute the instructions to: acquire first information about a first state of a target object at a first position, second information about a second state of the target object at a second position, third information about a third state of the target object in an action that is performed between the first position and the second position, and fourth information about an area where the target object is placeable in an inclined state; and generate an action plan of a robot related to the target object so that an action to be performed on the target object from the first information to the second information acquired by the acquisition means is performed via the third information, wherein the action plan includes an action for placing the target object in the area. 101 Analysis – Step 1: Statutory Category – Yes The claim recites a machine. The claim falls within one of the four statutory categories. MPEP 2106.03 relates. 101 Analysis – Step 2A Prong One Evaluation: Judicial Exception – Yes – Mental Processes In Step 2A, Prong one of the 2019 Patent Eligibility Guidance (PEG), a claim is to be analyzed to determine whether it recites subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) mental processes, and/or c) certain methods of organizing human activity. The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes”. MPEP 2106.04(a)(2)(III) relates. The claim recites limitations of generating an action plan for a particular purpose. These limitations, as drafted, are the performance of mental processes but for the recitation of “a processing device comprising: a memory … and a processor”. That is, other than reciting “a processing device comprising: a memory … and a processor” nothing in the claim elements precludes the limitations from being performed in the human mind, with or without the aid of pen and paper. The mere nominal recitation of “a processing device comprising: a memory … and a processor” does not take the claim limitations out of the mental processes grouping. For example, a person might readily come up with an action plan based on known facts (the recited states) with or without the aid of pen and paper. Examiner furthermore notes that presently the nature of “acquire” is not provided and may also be a part of the abstract idea, a person merely imagining or otherwise coming up with the state information in their head or with the aid of pen and paper. Thus, the claim recites mental processes. 101 Analysis – Step 2A Prong Two Evaluation: Practical Application – No In Step 2A, Prong two of the 2019 PEG, a claim is to be evaluated whether, as a whole, it integrates the recited judicial exception into a practical application. As noted in MPEP 2106.04(d), it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. The courts have indicated that additional elements such as: merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.” The Office submits that the foregoing underlined limitation (s) recite additional elements that do not integrate the recited judicial exception into a practical application. The claim recites additional elements or steps of the activities being functions of “a processing device comprising: a memory … and a processor”. The use of a computing device merely describes how to generally perform the generation of an action plan using a generic device, i.e. a computer and is recited at a high level of generality and is merely automating or performing the functions. With respect to the remaining underlined limitations reciting functions of “acquire”, this is insignificant extra-solution activity of mere data gathering and outputting. MPEP 2106.05(g)(3) relates. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. 101 Analysis – Step 2B Evaluation: Inventive Concept – No In Step 2B of the 2019 PEG, a claim is to be evaluated as to whether the claim, as a whole, amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. MPEP 2106.05 relates. As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than mere instructions to apply the exception using a generic computer component(s). The same analysis applies here in 2B, i.e., mere instructions to apply an exception on a generic computer or computer component(s) cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B. Under the 2019 PEG, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B. Again, the functions of “acquire …” is insignificant extra-solution activity of mere data gathering which is insufficient for both Step 2A Prong Two and Step 2B considerations. MPEP 2106.05(g)(3) relates. The specification does not provide any indication that the computing device is anything other than a conventional computer. MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures I, LLC v. Symantec Corp. , 838 F.3d 1307, 1321 (Fed. Cir. 2016), TLI Communications LLC v. AV Auto. LLC , 823 F.3d 607, 610 (Fed. Cir. 2016), and OIP Techs., Inc., v. Amazon.com, Inc. , 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere receipt or transmission of data over a network is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). Accordingly, a conclusion that the providing step is well-understood, routine, conventional activity is supported under Berkheimer. Thus, the claim is ineligible. With respect to the dependent claims indicated as rejected above (all but Claim 2), the claims merely recite additional details to the mental processes already recited or circumstantial items to the devices involved. In other words, the additional limitations merely recite details which are categorically already addressed above. For example, Claim 3 recites the storage of a neural network, but it is completely tangential to the other limitations of the claims and is recited at a high level of generality. In other words, the other limitations of the claims do not interact with the trained neural network, and the trained neural network does not interact with (e.g. perform any function of) the other limitations of the claims, and the neural network itself effectively recites “was trained using the known information”. With respect to claim 4, the limitations amount to making an unspecific judgement call using unspecific criteria and means and generating the action plan in accordance thereof. In other words, performing the application of basic judgement perfectly within the realm of human capacity. Examiner notes that Claim 5 is not included under the present interpretation provided above in response to the 112(b) rejection, wherein operation based on the action plan is clearly recited. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15 AIA Claim s 1 – 7 are rejected under 35 U.S.C. 102( a)(1) and 102(a)(2 ) as being anticipated by Diankov et al. (US 20210154839 A1) . Regarding Claim 1, Diankov teaches: A processing device ( See at least [0071] “The techniques of the present disclosure can be implemented in a special purpose computer or data processor that is programmed, configured, or constructed to execute one or more instructions on various computers” ) comprising: a memory configured to store instructions ( See at least [0071] “Instructions for executing computer- or controller-executable tasks can be stored in or on any suitable computer-readable medium” ) ; and a processor configured to execute the instructions ( See again [0071] above, in particular “a special purpose computer or data processor” ) to: acquire first information ( The term “information” is not defined. A state as already provided in the claim is information ) about ( In the interest of compact prosecution, Examiner notes that the phrase “about” is especially broad and merely indicates a relationship, no matter how vague ) a first state ( The term “state” is undefined. A position as already provided in the claim is a state ) of a target object ( See at least operation object 112 ) at a first position ( See at least start location 114 (full noun phrase being “start location 114 of the operation object 112”) ) , second information about a second state of the target object at a second position ( See at least task location 116 ) , third information about a third state of the target object ( See at least shift location 118 ) in an action that is performed between the first position and the second position ( Inherent as illustrated by line Figure 4B of control sequence 474. See also [0123] for further clarity as needed, “shifting the operation object 112 at the shift location 118 (changing the grip location), moving the operation object 112 from the start location 114 to above the task location 116, as necessary, via a shift location 118, lowering the operation object 112” ) , and fourth information about an area where the target object is placeable in an inclined state ( See at least [0134] “In addition, the robotic system 100 can derive the control sequence based on … the information regarding the operation object 112 in a storage container 450 placed at the task location 116 (e.g., a box, a bin, and the like)”. The phrase “where the target object is placeable in an inclined state” does not appear to particularly limit the term “area”, as it merely states “placeable”, or in other words, the capacity to be placed in such a manner. There is no actual requirement that the target object actually be placed in an inclined state. Relatedly, there does not appear to be any particular disclosure regarding this limitation further clarifying/describing placement in an inclined state, what is considered an inclined state, etc. ) ; and generate an action plan of a robot related to the target object ( See variously recited control sequence, for example and specific control sequence 422, control sequence 424, control sequence 472, or control sequence 474 ) so that ( Mere indication of intent or purpose of the preceding, and not a positively recited limitation. This is made especially clear by Claim 2 as it is apparent therefrom that no execution and control is actually performed here ) an action to be performed on the target object from the first information to the second information acquired by the acquisition means is performed via the third information ( In the interest of compact prosecution, see at least Figure 4B illustrating control sequence 474 via the associated dashed line wherein an operation object 112 is taken from start location 114 to task location 116 via shift location 118 and the associated grasping and motion trajectory ) , wherein the action plan includes an action for placing the target object in the area ( See at least [0055] “moving the operation object to a placement area” and Figures 4A and 4B ) . Regarding Claim 2, Diankov teaches: The processing device according to claim 1, wherein the processor is configured to execute the instructions to control the robot based on the action plan ( See at least Figure 4B illustrating control sequence 474 via the associated dashed line wherein an operation object 112 is taken from start location 114 to task location 116 via shift location 118 and the associated grasping and motion trajectory ) . Regarding Claim 3, Diankov teaches: The processing device according to claim 1, wherein the memory stores a trained neural network whose parameters are decided based on labeled data ( Examiner notes that in light of Applicant’s apparent broad use of the term which appears to be broader that the normal plain meaning of the term, Diankov appears to read on the claim as it is of a combination of data. In the interest of compact prosecution, see Shibata (US 20180099408 A1) in the Conclusion section which might be relied upon for a rejection (in the alternative or actual) under 35 USC 103 should Applicant dispute the claim interpretation above or otherwise amend the claim to more clearly specify the nature of “labeled” ) in which the first information, the second information, the third information, and the fourth information are designated as inputs and the action plan is designated as an output ( See at least [0060] “The robotic system can further execute a control sequence for executing each task according to a location, a pose, a height, and a confidence measure of the operation object, or a combination thereof, and/or a location and a pose of the robot, or a combination thereof. Such a control sequence can be created or derived by machine learning such as motion planning and deep learning” ) . Regarding Claim 4, Diankov teaches: The processing device according to claim 1, wherein the processor is configured to execute the instructions to whether or not to change a state of the target object based on the first information, the second information, the third information, and the fourth information and generates the action plan including an action for ( Indication of intent or purpose and not a positively recited limitation ) changing the state of the target object in the area in a case where it is determined to change the state of the target object ( See at least [0142] “Therefore, when a new pose or grip location different than the current grip location/orientation increases a filling rate of the storage container 450, the robotic system 100 can change remaining portions of the control sequence 472 (corresponding to the first control sequence 422 or the second control sequence 424 in FIGS. 4A and 4B) and create an updated control sequence including an operation of shifting the operation object 112 so as to be a pose optimized for storage” ) . Regarding Claim 5, Diankov teaches: A robot system comprising: the processing device according to claim 1; and the robot configured to be controlled based on the action plan generated by the processing device ( See at least transfer unit 104 ) . Regarding Claims 6 and 7, the claims are directed to effectively the same subject matter as Claim 1 with respect to the application of prior art. The claims are therefore rejected under the same logic as Claim 1 above . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Shibata (US 20180099408 A1) which discloses deep learning, labelled data, the use of an input-output pair in training of such context, and all within the context of robotic gripping control (See e.g. [0068], [0071], [0096], [0098], [0099], and [0100]). Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW C GAMMON whose telephone number is (571)272-4919. The examiner can normally be reached M - F 10:00 - 6:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ADAM MOTT can be reached on (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MATTHEW C GAMMON/Examiner, Art Unit 3657 /ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657 Application/Control Number: 19/137,690 Page 2 Art Unit: 3657 Application/Control Number: 19/137,690 Page 3 Art Unit: 3657 Application/Control Number: 19/137,690 Page 4 Art Unit: 3657 Application/Control Number: 19/137,690 Page 5 Art Unit: 3657 Application/Control Number: 19/137,690 Page 6 Art Unit: 3657 Application/Control Number: 19/137,690 Page 7 Art Unit: 3657 Application/Control Number: 19/137,690 Page 8 Art Unit: 3657 Application/Control Number: 19/137,690 Page 9 Art Unit: 3657 Application/Control Number: 19/137,690 Page 10 Art Unit: 3657 Application/Control Number: 19/137,690 Page 11 Art Unit: 3657 Application/Control Number: 19/137,690 Page 12 Art Unit: 3657 Application/Control Number: 19/137,690 Page 13 Art Unit: 3657 Application/Control Number: 19/137,690 Page 14 Art Unit: 3657 Application/Control Number: 19/137,690 Page 15 Art Unit: 3657 Application/Control Number: 19/137,690 Page 17 Art Unit: 3657 Application/Control Number: 19/137,690 Page 18 Art Unit: 3657
Prosecution Insights