DETAILED ACTION
Remarks
This non-final office action is in response to the application filled on 07/11/2025. Claims 1-20 are pending and examined below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for domestic benefit under 35 U.S.C. 119 (e). The provisional application No. 63/713,798, was filed on 10/30/2024.
Information Disclosure Statement
As of date of this action, IDS filled has been annotated and considered.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION. —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding claim 1 (and similarly claim 11 and 20), which recites “a first trained machine learning model” line 6 is unclear and indefinite since first trained machine learning model is recited earlier on line 4. It is not clear both first trained machine learning model is same or different.
Dependent claim(s) 2-10 and 12-19 is/are also rejected because they do not resolve their parent deficiencies.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 9, 11, 18 and 20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by US 2021/0347040 (“Nagarajan”).
Regarding claim 1 (and similarly claim 11 and 20), as best understood in view of indefiniteness rejection explained above, Nagarajan discloses a computer-implemented method for controlling a robot to grasp an object (see at least fig 8, see also [0003], where “process sensor data (e.g., vision data), using a trained machine learning model, to generate output that defines one or more grasp regions and, for each of the one or more grasp regions, a corresponding semantic indication associated with the grasp region.”), the method comprising:
receiving sensor data from one or more sensors (see at least [0003], where “The sensor data is generated by one or more sensors of a robot, and captures features of an object to be grasped by the robot (and optionally captures features of additional environmental object(s)). For example, the sensor data can include vision data that is generated by a vision component of a robot, and that captures an object to be grasped by the robot.”);
generating, based on the sensor data and using a first trained machine learning model, one or more grasp poses (see at least [0053], where “the grasp strategy can be selected…from a plurality of candidate grasp strategies. Each candidate grasp strategy defines a different group of one or more values that influence performance of a grasp attempt in a manner that is unique relative to the other grasp strategies.”; see also fig 7, [0061], [0085] and [0113]);
selecting, from the one or more grasp poses and using a first trained machine learning model, one or more filtered grasp poses (see at least fig 9, block 956);
generating, based on the one or more filtered grasp poses, a grasping plan (see at least fig 9, block 958); and
causing the robot to grasp the object based on the grasping plan (see at least fig 9, block 960).
Regarding claim 2, Nagarajan further discloses a method comprising generating, based on the sensor data, an object geometry embedding, wherein generating the one or more grasp poses and selecting the one or more filtered grasp poses are based on the object geometry embedding (see at least [0112], [0067] and [0006]).
Regarding claim 3, Nagarajan further discloses a method wherein generating the object geometry embedding comprises: generating, based on the sensor data and using an encoder, object geometry data (see at least fig 10, [0004], [0060] and [0067]); and
generating, based on the object geometry data, the object geometry embedding (see at least [0057]).
Regarding claim 9 (and similarly claim 18), Nagarajan further discloses a method wherein generating the grasping plan comprises determining, for each filtered grasp pose included in the one or more filtered grasp poses, at least one of kinematic feasibility or one or more collision constraints (see at least [0036] and [0058]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0347040 (“Nagarajan”), as applied to claim 1 above, and further in view of US 2024/0386529 (“Yang”).
Regarding claim 4, Nagarajan does not disclose claim 4. However, Yang discloses a method wherein the first trained machine learning model comprises a denoising diffusion probabilistic model (DDPM) (see at least [0073], where “outputs generated by the pre-trained model 130 and the domain-specific model 140 can be interpreted in the framework of energy based analysis of denoising diffusion probability models.”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Nagarajan to incorporate the teachings of Yang by including the above feature for increasing training stability by adding and removing noise.
Claim(s) 5, 6 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0347040 (“Nagarajan”), as applied to claim 1 and 11 above, and further in view of US 2024/0273261 (“Brehmer”).
Regarding claim 5 (and similarly claim 13), Nagarajan does not disclose claim 5. However, Brehmer discloses a method wherein generating the one or more grasp poses comprises, for each iteration included in one or more iterations of a reverse diffusion technique (see at least [0042], where “Each of the trajectory points 110 represent a state based on an action (e.g., a position of the structure 102 based on a particular movement of the structure 102). The original trajectory can have many different states based on a diffused or noisy environment in which the structure 102 can move to many different possible points along a trajectory in order to receive the object 104 (e.g., grasp the object 104). Based on performing a denoising 108 flow, the model can iteratively refine the trajectory such that, in a new state represented by trajectory points 112, the trajectory becomes more focused or refined. The trajectory can be further refined as shown by trajectory points 114 in further iterations. An opposite flow can cause diffusion 116 in which a more specific trajectory (e.g., the trajectory represented by trajectory points 114 or the trajectory represented by trajectory points 112) can be diffused and become random.”):
generating, based on a time step and using an encoder, a time step embedding (see at least [0033], [0044] and [0051]);
generating, based on a noisy grasp pose and using a third machine learning model, a noisy grasp pose embedding (see at least [0049], where “FIG. 4 provides two sets of images 400 that show the forward diffusion process (which is fixed) and the reverse diffusion process (which is learned) of a diffusion model.”); and
generating, based on an object geometry embedding, the time step embedding, and the noisy grasp pose embedding, a predicted noise (see at least [0067], [0086] and [0110]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Nagarajan to incorporate the teachings of Brehmer by including the above feature for providing continuous, multimodal, and collision free grasp generation.
Regarding claim 6, Nagarajan further discloses a method of claim 5, wherein at least one of the encoder or the third machine learning model comprises a multilayer perceptron (see at least [0115] and [0096]).
Claim(s) 7, 8, 14, 15 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0347040 (“Nagarajan”), as applied to claim 1 and 11 above, and further in view of US 2023/0321821 (“Ku”).
Regarding claim 7 (and similarly claim 14), Nagarajan does not disclose claim 7. However, Ku discloses a method wherein selecting the one or more filtered grasp poses comprises:
generating, based on the one or more grasp poses and using the first trained machine learning model, one or more predicted grasp pose scores (see at least fig 1, block S500);
ranking, based on the one or more predicted grasp pose scores, each grasp pose included in the one or more grasp poses to generate one or more ranked grasp poses (see at least [0105], where “selecting a set of final grasp proposals based on the grasp scores can include ranking the grasp proposals based on the grasp scores; and selecting the grasp proposals with the highest grasp scores (e.g., up to the top 3, up to the top 5, up to the top 10, up to the top 15, up to the top 20, etc.).”); and
selecting, based on the one or more ranked grasp poses, the one or more filtered grasp poses (see at least [0106]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Nagarajan to incorporate the teachings of Ku by including the above feature for providing most successful grasp strategy for picking an object.
Regarding claim 8 (and similarly claim 17), Nagarajan does not disclose claim 8. However, Ku further discloses a method wherein the one or more filtered grasp poses are selected based on one or more highest scores associated with the one or more filtered grasp poses or based on a threshold (see at least [0108], where “the grasp proposal can be selected based on the grasp score (e.g., highest grasp score)”).
Regarding claim 15, Nagarajan does not disclose claim 15. However, Ku further discloses a wherein the one or more predicted grasp scores include at least one of a continuous value between zero and one representing a confidence in a successful grasp, a grasp success probability, or a binary grasp success or failure prediction (see at least [0032], [0092-93] and [0104]).
Claim(s) 10 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0347040 (“Nagarajan”), as applied to claim 1 and 11 above, and in view of US 2022/0402134 (“Pidaparthi”), and further in view of US 2019/0017839 (“Eyler”).
Regarding claim 10 (and similarly claim 19), Nagarajan does not disclose claim 10. However, Pidaparthi discloses a method comprising: performing, based on grasp data, one or more operations to train a first untrained machine learning model to generate the first trained machine learning model, wherein the first trained machine learning model is trained to generate a predicted noise (see at least [0041], where “the implementing the machine learning process to determine the state estimation model may include implementing the machine learning process to model a noise profile in the system (e.g., noise generated by operation of the robotic arm to move an items, such as a sway in an item during movement; noise generated by the vision system when capturing the sensor data, etc.).”; see also [0118], where “the modelling the noise includes performing a machine learning process to train a noise profile (e.g., a noise model).”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Nagarajan to incorporate the teachings of Pidaparthi by including the above feature for providing a smoother optimization by using cleaner and consistent model.
Nagarajan in view of Pidaparthi does not disclose the following limitations:
performing, based on the grasp data, the first trained machine learning model, and a simulator, one or more operations to generate augmented grasp data; and
performing, based on the augmented grasp data, one or more operations to train a second untrained machine learning model to generate the second trained machine learning model, wherein the second trained machine learning model is trained to generate a predicted grasp pose score.
However, Eyler discloses a method comprising: performing, based on the grasp data, the first trained machine learning model, and a simulator, one or more operations to generate augmented grasp data (see at least [0123], where “once the augmented reality transportation system 106 generates a passenger pickup location route and/or a driver pickup location route, the augmented reality transportation system 106 generates an augmented reality element, as shown by act 212 of the sequence 200 in FIG. 2.”; Nagarajan discloses grasp data. Eyler discloses augmented pick-up location. So, it would be obvious to generate augmented grasp data utilizing Augmented pick-up location process disclosed by Eyler.); and
performing, based on the augmented grasp data, one or more operations to train a second untrained machine learning model to generate the second trained machine learning model, wherein the second trained machine learning model is trained to generate a predicted grasp pose score (see at least [0099], where “the augmented reality transportation system 106 scores various locations based on factors such as passenger ratings, passenger waiting time, proximity to the passenger, etc., to score pickup and/or drop-off locations.”; see also [0026], [0098] and [0100]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Nagarajan in view of Pidaparthi to incorporate the teachings of Eyler by including the above feature for providing a model that is compatible for dynamic environments and creates context awareness.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0347040 (“Nagarajan”), as applied to claim 11 above, and further in view of US 2024/0083021 (“Power”).
Regarding claim 12, Nagarajan does not disclose claim 12. However, Power discloses a system wherein generating the grasping plan comprises selecting a filtered grasp pose included in the one more filtered grasp poses that is associated with a lowest-cost trajectory (see at least [0072], where “The lowest cost trajectory may be selected and executed for the first pose and grasp change.”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Nagarajan to incorporate the teachings of Power by including the above feature for providing optimized trajectory for increasing efficiency.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2021/0347040 (“Nagarajan”), as applied to claim 11 above, and further in view of US 2022/0309672 (“Cherian”).
Regarding claim 16, Nagarajan does not disclose claim 16. However, Cherian discloses a system wherein the one or more grasp poses include a rigid body transformation in the Special Euclidean group in three dimensions (SE (3)), and wherein the rigid body transformation includes a rotation component in the Special Orthogonal group in three dimensions (SO (3)) and a translation component in three-dimensional Euclidean space (see at least [0060], [0065], [0090] and [0128-129]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Nagarajan to incorporate the teachings of Cherian by including the above feature for planning diverse, precise, and stable grasp poses on novel objects by providing complete mathematical framework to represent a rigid body in 3D space.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOHANA TANJU KHAYER whose telephone number is (408)918-7597. The examiner can normally be reached Monday - Thursday, 7 am-5.30 pm, PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached at 5712703976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SOHANA TANJU KHAYER
Primary Examiner
Art Unit 3657C
/SOHANA TANJU KHAYER/ Primary Examiner, Art Unit 3657
Prosecution Insights