Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
Claims 1-2 and 4-7 are currently pending and are being hereby examined herein.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Response to Amendment/Remarks
The term “previous office action” refers to the non-final rejection dated 2 April 2025.
The drawing objections from the previous office action have been withdrawn due to amendment. All but one objection to the specification from the previous office action have been withdrawn, the objection for paragraph [0044] is clarified below. All claim objections in the previous office action have been withdrawn. The interpretation of “a robotic device control unit” under 35 U.S.C. 112(f) from the previous office action has been withdrawn due to the amendment. The rejections under 35 U.S.C. 112(b) from the previous office action have been withdrawn due to amendments.
The abstract idea rejection under 35 U.S.C. 101 from the previous office action has been withdrawn due to amendments. Additionally, Examiner has withdrawn the rejections for non-statutory subject matter (signal per se) for Claims 6 and 7; however, Examiner has issued a new means of rejection under 35 U.S.C. 112(b) for Claims 6 and 7, necessitated by amendment, because it is indefinite how a method includes a computer program / which step of the method of Claim 1 is being referenced by the “wherein…” (see below).
Applicant argues that Beckman does not disclose “randomly selecting one control vector of the plurality of control vectors according to the adjusted probability distribution for performing the specified tasks in a current control scenario”; however, the argument is not persuasive. In Applicant’s original disclosure, the term randomly occurs once in the abstract (“randomly selects a control vector according to the probability distribution for performing the task in a current control scenario”) twice in the specification (“randomly selecting a control vector according to the probability distribution for performing the task in a current control scenario”, “control vector is randomly selected according to the probability distribution for performing the task in a current control scenario”) and once in the claims (“randomly selecting one of the plurality of control vectors according to the probability distribution for performing the task in a current control scenario”). Every mention of “randomly” is accompanied with “probability distribution”. Therefore, Examiner sees no indication of truly “random” selection, for every mention of “randomly” the selection is weighted by the results of the ML model; therefore, Examiner has interpreted “randomly selecting one control vector of the plurality of control vectors according to the adjusted probability distribution…” under the broadest reasonable interpretation to include selecting control vector of the plurality of control vectors in accordance with the updated machine learning model which is the same way one of ordinary skill in the art would interpret this (paragraph [0051] and [0055] of the specification: “the probability distribution is adjusted over time…such that control vectors that resulted in control sequences were met for the target conditions…are more likely to be sampled.”; “over time the ML model increasingly selects control vectors that will result in sequences of primitives for which the machine-learning model (according to its past experiences) expects to fulfill the conditions of the target metrics”). As Beckman is an iterative machine learning method, this limitation is met. In addition, the terms “each control vector” is interpreted to include includes a subset of at least two (a plurality of) possible control vectors (i.e., each selected control vector of a plurality of control vectors): depending on the number of primitives, there is an infinite number of control vectors and Examiner sees no indication that the specification requires each and every one of them to be completed which would be consistent with how one of ordinary skill in the art understands machine learning (paragraph [0050]: “The ML model samples the control vectors from a space of control vectors.”); therefore, the “randomly” changing of Beckman is merely expanding to include more of the plurality of control vectors in accordance with the machine learning model, however, the “randomly” expanding is not even required to read-on the current claims in view of the specification; all that is required in selection in accordance with the machine learning model.
Furthermore, Applicant argues that Beckman teaches away citing (col. 2, lines 33-37). Examiner agrees that Beckman suggests starting without truly random-ness / in virtual simulation, but then includes multiple “real-world” iterations with machine learning, which is the area Examiner cited; the current Application also does not suggest “random selection of a control policy” but selection in accordance with a machine learning model and in accordance with rules, similar to Beckman. Therefore, Beckman is not teaching away, but teaching a similar solution, and the argument is not persuasive.
Specification
The disclosure is objected to because of the following informality:
Newly amended paragraph [0044] states that in the nominal case, Special case 1, and Special case 2 the barcode is not at/on the upper surface; and in Special case 3 and Special case 4 the barcode is at/on the upper surface. This appears to be the inverse of how it should be stated. The reorientation would be required when the barcode is not at/on the upper surface since the camera appears to be taking pictures of the upper surface so on the upper surface is when the barcode can be read.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 6-7 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention. These claims are method claims, but claim a computer program. It would be indefinite to one of ordinary skill if the computer program is required, as a computer program is not a step in a method. Appropriate corrections are required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2 and 4-7 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Beckman et al. (U.S. Patent No. 10,800,040, hereinafter, “Beckman”).
Regarding Claim 1, Beckman teaches: A method (see at least FIG. 3 and the patent generally: a method of controlling a real-world robot is described):
controlling, for each control vector of each control vector of the plurality of control vectors indicates which of the first actions is to be performed for a respectively observed control situation of a specified task (See at least column 2 lines 41-46, column 2 lines 54-55, column 4 lines 1-3, and FIG. 1: completing a task (i.e., a respectively observed control situation) involves several steps and a control policy (i.e., a control vector) specifies the actions required to complete each of the steps, the robot is guided through the sequence of actions needed to perform the task by its controller (i.e., controlling…the robot device to perform a first sequence of first actions), the robot can iterate through a number of real-world trial runs of the task (i.e., the task is completed for each control vector from a plurality of control vectors));
determining, for the [[a]] specified task (See at least column 11 lines 48-56, column 11 lines 64-66, column 12 lines 2-11, and FIG. 3: the robot performance is evaluated either by a human or a programmatic module to indicate its level of success, and the performance can be evaluated to determine if it complies with safety regulations or standards, and the condition of the objects can be evaluated to see if they’re in good condition);
adjusting a probability distribution of the plurality of control vectors by increasing a correspondingthe control vectors for which the specified task resulted in have satisfied at least one target condition (See at least column 9 lines 29-47, column 11 lines 60-63, column 12 lines 8-11, and FIG. 3: the machine learning system refines the policy (i.e., it is adjusting a probability distribution) based on the evaluation of the real-world performance using a predefined success metric (i.e., satisfied at least one target condition) and continues to select versions that perform best);
randomly selecting one control vector of the plurality of control vectors according to the adjusted probability distribution for performing the specified task in a current control scenario (See at least column 9 lines 29-67 and column 10 lines 1-3: the machine learning system can make different random adjustments to the parameters, test them, and then continue randomly adjusting slightly / selecting the most successful version, in accordance with the ML model, until desired performance is reached);
controlling the robotic device to perform a second sequence of second actions, wherein the randomly selected one control vector of the plurality of control vectors indicates which of the second actions is to be performed for [[a]] the respectively observed control situation of the specified task; (See at least column 9 lines 29-67 and column 10 lines 1-3: there can be multiple generations and iterations of the control policy, the control policy includes parameters that dictate what actions or sequence of actions the robotic system should take) and
performing the specified task using the robotic device to pick up an object from a container according to the randomly selected one control vector (see at least column 2 lines 39-54, column 4 lines 40-61, and column 10 lines 17-21: transferring objects to and from storage structures is an application of the machine learning techniques).
Regarding Claim 2, Beckman teaches all the limitations of claim 1. Additionally, Beckman teaches wherein the randomly selected one control vectorfirst actions for a control situation observed in each case for at least some of the first actions (See at least column 9 lines 6-10, column 9 lines 64-67, and column 10 lines 1-3: the control policy (i.e., control vector) indicates parameters about the actions).
Regarding Claim 4, Beckman teaches all the limitations of claim 1. Additionally, Beckman teaches: wherein the adjustment of the probability distribution is carried out using a gradient-free optimization process (See at least column 8 lines 10-26: the machine learning system can use black-box optimization (i.e., gradient-free optimization) techniques such as evolution strategies).
Regarding Claim 5, Beckman teaches all the limitations of claim 1. Additionally, Beckman teaches: A robotic device, comprising: a processor (See at least column 5 lines 30-49, column 6 lines 6-30, column 11 lines 1-10, FIG. 2B, and the patent generally: the robotic control system 220 can include one of more computers, processors and memory are sub-components of the computers, this has the training technique; the controller 250 contains one or more processors, includes one or more physical data storage devices, and has instructions for controlling the robotic system).
Regarding Claim 6, Beckman teaches all the limitations of claim 1. Additionally, Beckman teaches: wherein a [[A]] computer program comprises the (See at least column 5 lines 30-49, column 6 lines 6-30, column 11 lines 1-10, FIG. 2B, and the patent generally: the controller 250 contains one or more processors, one or more physical data storage devices that store the control policy, and has executable instructions for controlling the robot; the robotic control system 220 can include one of more computers, processors are sub-components of the computers, it generates the robotic control policy).
Regarding Claim 7, Beckman teaches all the limitations of claim 6. Additionally, Beckman teaches: wherein the computer program is non-transitory computer-readable medium (See at least column 5 lines 30-49, column 6 lines 6-30, and FIG. 2B: the controller 250 includes one or more physical data storage devices that store the control policy; the robotic control system 220 can include one of more computers, memory is a sub-component of the computers).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDRA ROBYN MORFORD whose telephone number is (571)272-6109. The examiner can normally be reached Monday - Friday 8:00 AM - 4:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Worden can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.R.M./Examiner, Art Unit 3658 /JASON HOLLOWAY/ Primary Examiner, Art Unit 3658