The present application, filed on or after 16 March 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to Applicant’s submission filed on 26 May 2026. THIS ACTION IS NON-FINAL.
In response to the restriction requirement, Applicant’s election without traverse of the instant application in the reply filed on 26 May 2026 is acknowledged.
Status of Claims
Claims 1-20 are pending.
Claims 1-10, 18-20 are withdrawn.
Claims 11-17 are rejected under 35 U.S.C. 112(b) as indefinite.
Claims 11-17 are rejected under 35 U.S.C. 103 as unpatentable.
Claim Rejections - 35 USC § 112
112(b) Rejection
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
A claim is indefinite if, when read in light of the specification, it fails to inform, with reasonable certainty, those skilled in the art about the scope of the invention. Nautilus, Inc. v. Biosig Instruments, Inc., 110 USPQ.2d 1688, U.S. Supreme Court (2014).
Claims 11-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding claim 11, “… the first / second machine learning model being trained …”, it is not clear if a training step is included in an otherwise inference-only method claim, the claim is indefinite. For the purpose of applying prior art, this limitation is construed to be “…the first / second machine learning model being pre-trained …”.
Regarding claims 12-17, which depend on above rejected claim 11, are rejected for the same reason.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11, 13-15, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Akkaya et al, “Solving rubik’s cube with a robot hand”, arXiv:1910.07113v1 [cs.LG] 16 Oct 2019 [hereafter Akkaya] in view of Lee et al, US-PGPUB NO.20210101286A1 [hereafter Lee].
With regards to claim 11, Akkaya teaches “A method comprising: determining a pose based at least on processing a plurality of images of a robot using a first machine learning model (Akkaya, FIG.1-3, FIG.10-12,
PNG
media_image1.png
980
729
media_image1.png
Greyscale
1 Introduction, ‘… The vision-based state estimator uses rendered scenes collected from the randomized simulations and learns to predict the pose as well as face angles of the Rubik’s cube using a convolutional neural network (CNN) …’), the first machine learning model being trained based at least on one or more rendered images corresponding to one or more first simulations of the robot (Akkaya, FIG.1-3, FIG.10-12,
PNG
media_image2.png
441
760
media_image2.png
Greyscale
5 Automatic Domain Randomization, ‘Instead of rolling out a policy, we use the ADR parameters to render images and use those to train the supervised vision state estimator’, ‘… We randomize simulator physics parameters such as geometry, friction, gravity, etc. See Section B.1 or details of their ADR parameterization …’, 6 Policy Training in Simulation, 7 State Estimation from Vision, ‘…Vision for pose and Giiker cube for face angels …);
generating an action based at least on processing the pose, a goal, and one or more previous states of the robot using a second machine learning model (Akkaya, FIG.1-3, FIG.11-12,
PNG
media_image3.png
450
769
media_image3.png
Greyscale
1 Introduction, ‘… The control policy receives observed robot sates and rewards from the randomized simulations and learns to solve them using a recurrent neural network and reinforcement learning …’, 2 Tasks, ‘… The goal of the block reorientation task is to rotate a block into a desired goal orientation …’), the second machine learning model being trained based at least on one or more second simulations of the robot (Akkaya, FIG.1-3, FIG.11-12,
PNG
media_image4.png
426
731
media_image4.png
Greyscale
6 Policy Training in Simulation) …”.
Akkaya does not explicitly detail “controlling the robot based at least on the action”.
However Lee teaches “controlling the robot based at least on the action (Lee, FIG.1, FIG.5, [0038] ‘…generate corresponding output that is utilized in control of the robot …’,
PNG
media_image5.png
936
723
media_image5.png
Greyscale
)”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Akkaya and Lee before him or her, to modify the simulation based ML models for robots action of Akkaya to include robot control as shown in Lee.
The motivation for doing so would have been to use trained policy model for robot control (Lee, Abstract).
With regards to claim 13, Akkaya in view of Lee teaches
“The method of claim 11, wherein the one or more rendered images are rendered based at least on at least one of different camera parameters, different visual effects, or performing one or more ray tracing operations (Akkaya, FIG.1-3, FIG.10-12, 1 Introduction, ‘.. We use automatic domain randomization (ADR) to generate a growing distribution of simulations with randomized parameters and appearance …’, 5 Automatic Domain Randomization).”
With regards to claim 14, Akkaya in view of Lee teaches
“The method of claim 11, wherein the first machine learning model is further trained based at least on one or more augmented images, and the one or more augmented images are generated by performing one or more operations to augment the one or more rendered images based at least on at least one of a lighting augmentation, a texture augmentation, or a geometry augmentation (Akkaya, FIG.1-3, FIG.10-12, 1 Introduction, ‘.. We use automatic domain randomization (ADR) to generate a growing distribution of simulations with randomized parameters and appearance …’, 5 Automatic Domain Randomization).”
With regards to claim 15, Akkaya in view of Lee teaches
“The method of claim 11, wherein the second machine learning model is trained by performing one or more operations to simulate the robot in a plurality of simulations, and the plurality of simulations are based at least on at least one of different physics parameters or different non-physics parameters (Akkaya, FIG.1-3, FIG.10-12, 5 Automatic Domain Randomization, ‘… We randomize simulator physics parameters such as geometry, friction, gravity, etc. See Section B.1 for details of their ADR parameterization …’).”
With regards to claim 17, Akkaya in view of Lee teaches
“The method of claim 15, wherein the second machine learning model is trained by further performing one or more automatic domain randomization operations to determine ranges of the at least one of different physics parameters or different non- physics parameters (Akkaya, FIG.1-3, FIG.10-12, 5 Automatic Domain Randomization, ‘… We randomize simulator physics parameters such as geometry, friction, gravity, etc.’, Appendices B Randomizations,
PNG
media_image6.png
124
748
media_image6.png
Greyscale
).”
The combined teaching described above will be referred as Akkaya + Lee hereafter.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Akkaya et al, “Solving rubik’s cube with a robot hand”, arXiv:1910.07113v1 [cs.LG] 16 Oct 2019 [hereafter Akkaya] in view of Lee et al, US-PGPUB NO.20210101286A1 [hereafter Lee] and Lin et al., “Multi-view for multi-level robotic scene understanding”, arXiv:2103.13539v1 [cs.RO] 25 Mar 2021 [hereafter Lin].
With regards to claim 12, Akkaya + Lee teaches
“The method of claim 11”.
Akkaya + Lee does not explicitly detail “wherein the first machine learning model generates, for at least one image included in the plurality of images, a bounding shape, a segmentation, and one or more key points associated with at least one of the robot or an object that the robot interacts with in the at least one image, and the determining the pose comprises: determining one or more three-dimensional (3D) positions based at least on the one or more key points associated with the at least one of the robot or the object; and determining the pose based at least on a registration of the one or more 3D positions against one or more models of the at least one of the robot or the object”.
However Lin teaches “wherein the first machine learning model generates, for at least one image included in the plurality of images, a bounding shape, a segmentation, and one or more key points associated with at least one of the robot or an object that the robot interacts with in the at least one image (Lin, FIG.1-6,
PNG
media_image7.png
297
823
media_image7.png
Greyscale
II Related Work, ‘…incorporating bounding box keypoint …’, III Approach, C Multi-view object pose fusion,
), and the determining the pose comprises:
determining one or more three-dimensional (3D) positions based at least on the one or more key points associated with the at least one of the robot or the object (Lin, FIG.1-6,
); and
determining the pose based at least on a registration of the one or more 3D positions against one or more models of the at least one of the robot or the object (Lin, FIG.1-6,
PNG
media_image8.png
271
820
media_image8.png
Greyscale
I Introduction, III Approach, IV Experimental Results)”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Akkaya + Lee and Lin before him or her, to modify the simulation based ML models for robots action of Akkaya + Lee to include more details of 3D processing as shown in Lin.
The motivation for doing so would have been for multi-level scene awareness (Lin, Abstract).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Akkaya et al, “Solving rubik’s cube with a robot hand”, arXiv:1910.07113v1 [cs.LG] 16 Oct 2019 [hereafter Akkaya] in view of Lee et al, US-PGPUB NO.20210101286A1 [hereafter Lee] and Makoviychuk et al., “Issac Gym: high performance GPU-based physics simulation for robot learning”, arXiv:2108.10470v2 [cs.RO] 25 Aug 2021 [hereafter Makoviychuk].
With regards to claim 16, Akkaya + Lee teaches
“The method of claim 15”.
Akkaya + Lee does not explicitly detail “wherein the plurality of simulations are performed in parallel using one or more graphics processing units (GPUs)”.
However Makoviychuk teaches “wherein the plurality of simulations are performed in parallel using one or more graphics processing units (GPUs) (Makoviychuk, FIG.1-3,
PNG
media_image9.png
338
660
media_image9.png
Greyscale
1 Introduction, 2 Background)”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Akkaya + Lee and Makoviychuk before him or her, to modify the simulation based ML models for robots action of Akkaya + Lee to include GPU simulation as shown in Makoviychuk.
The motivation for doing so would have been for Robot learning (Makoviychuk, Abstract).
Additional Relevant Art
The prior art made of record is considered pertinent to applicant’s disclosure and is recorded on Form PTO-892. Applicant is required under 37 C.F.R. § 1.111 (c) to consider these references fully when responding to this action, with particular attention paid to:
He et al, US-PATENT NO.10713794B1 [hereafter He] shows image for training ML models.
Peng, et al., “Sim0to-real transfer of robotic control with dynamics randomization”, ICRA 2018 [hereafter Peng] shows using simulation for robot training.
Bousmalis, et al., “Using simulation and domain adaptation to improve efficiency of deep robotic grasping”, ICRA 2018 [hereafter Bousmalis] shows using simulation to improve robot efficiency.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSU-CHANG LEE whose telephone number is 571-272-3567. The fax number is 571-273-3567.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas, can be reached 571-272-2589.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSU-CHANG LEE/
Primary Examiner, Art Unit 2128
Prosecution Insights