DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation Note
Regarding instant claim 11, Examiner notes that U.S. Patent Application 17/981,770 contains a similarly-worded limitation in claim 21. Application 17/981,770 was appealed to the Patent Trial and Appeal Board (“the Board”) in Appeal No. 2024-003924, where the Board sua sponte entered a 35 U.S.C. §112 (d) rejection of claim 21 for failing to further limit its respective independent claim. Examiner notes that this 35 U.S.C. §112 (d) rejection was eventually overcome by amending claim 21 to recite “…wherein the system comprises at least one of….” Examiner recognizes that the decision in Appeal No. 2024-003924 is neither precedential nor binding on the examination corps. Nevertheless, Applicant may want to amend instant claim 11 in a manner consistent with the allowable version of the similar claims in Application 17/981,770. For example, instant claim 11 could be amended to read: “…wherein the one or more processors comprise at least one of….”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5–6, 11–12, 16–17 and 19 are rejected under 35 U.S.C. § 103 as being unpatentable over Grossmann et al. (U.S. 9,338,439 B2) in view of Theverapperuma et al. (U.S. 11,494,930 B2).
Regarding claim 1, Grossman discloses one or more processor comprising:
one or more circuits to: apply, to one or more features of a source image (one or more features of a source image construed as raw images) that depicts a scene using a first set of camera parameters (a first set of camera parameters construed as warped parameters), (Per Fig. 4 at step 410, Grossman discloses a pair of raw images. Then, he applies original warping parameters1 to determine whether the extent of epipolar misalignment is eminent. Grossman col. 7 line 51 – col. 8 line 4. [a]t 460, no adjustment is needed on the basis of the above epipolar misalignment determination. Note that while no adjustment may be needed on these grounds, additional testing (discussed below) may determine that adjustment of warping parameters is needed for other reasons.) based on a condition view image (a condition view image construed as rectified image(s)) associated with the source image, an epipolar geometric warping to determine a second set of camera parameters (a second set of camera construed as adjusted warped parameters). (Per Fig. 8, Grossman discloses adjusted warping parameters 820 after processing rectified images 350a–350c. Ibid. col. 10 line 61 – col. 11 line 16. This module may receive rectified images 350a . . . c and produce adjusted warping parameters 820.)
However, Grossman fails to specifically disclose generate, using a neural network, a synthetic image representing the one or more features and corresponding to the second set of camera parameters.
In related art, Theverapperuma discloses generate, using a neural network, a synthetic image (a synthetic image construed as a composite image) representing the one or more features (Per Fig. 10, Theverapperuma’s object detection module 1020 discloses a composite image regarding a 3D representation2 in his CNN model 1022. Theverapperuma col. 30 lines 37–56. The CNN depth model 1022 generates a 3D image based on the first 2D image and the second 2D image.) and corresponding to the second set of camera parameters (A second set of camera parameters construed as a second set of neural network layers in the trained model. See Applicant’s Spec ¶66. [a]nd the second set of camera parameters (e.g., associated with the condition view image 146) to a second neural network.). (Per Fig. 11, Theverappaeruma’s ML model 1120 discloses a second set of neural network layers to detect an object while determining whether inferences are incorrectly generated. Ibid. col. 33 lines 31–64. [a] 3D representation of a pile), a second set of neural network layers that perform object detection (e.g., detecting a pile of material in the single representation),)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Theverapperuma into the teachings of Grossman to provide techniques for volumetric estimation and dimensional estimation of an object. Ibid. col. 1 lines 19–25.
Regarding claim 12, Grossman discloses a system, comprising:
one or more processing units; and (Fig. 16, a CPU 1620)
one or more memory units (Fig. 16, a memory 1610) storing instructions that, when executed by the one or more processing units, cause the one or more processing units to execute operations comprising:
applying, to one or more features of a source image (one or more features of a source image construed as raw images) that depicts a scene using a first set of camera parameters (a first set of camera parameters construed as warped parameters), (Per Fig. 4 at step 410, Grossman discloses a pair of raw images. Then, he applies original warping parameters to determine whether the extent of epipolar misalignment is eminent. Grossman col. 7 line 51 – col. 8 line 4. [a]t 460, no adjustment is needed on the basis of the above epipolar misalignment determination. Note that while no adjustment may be needed on these grounds, additional testing (discussed below) may determine that adjustment of warping parameters is needed for other reasons.) based on a condition view image (a condition view image construed as rectified image(s)) associated with the source image, an epipolar geometric warping to determine a second set of camera parameters (a second set of camera construed as adjusted warped parameters). (Per Fig. 8, Grossman discloses adjusted warping parameters 820 after processing rectified images 350a–350c. Ibid. col. 10 line 61 – col. 11 line 16. This module may receive rectified images 350a . . . c and produce adjusted warping parameters 820.)
However, Grossman fails to specifically disclose generating, using a neural network, a synthetic image representing the one or more features and corresponding to the second set of camera parameters.
In related art, Theverapperuma discloses generating, using a neural network, a synthetic image (a synthetic image construed as a composite image) representing the one or more features (Per Fig. 10, Theverapperuma’s object detection module 1020 discloses a composite image regarding a 3D representation in his CNN model 1022. Theverapperuma col. 30 lines 37–56. The CNN depth model 1022 generates a 3D image based on the first 2D image and the second 2D image.) and corresponding to the second set of camera parameters (A second set of camera parameters construed as a second set of neural network layers in the trained model. See Applicant’s Spec ¶66. [a]nd the second set of camera parameters (e.g., associated with the condition view image 146) to a second neural network.). (Per Fig. 11, Theverappaeruma’s ML model 1120 discloses a second set of neural network layers to detect an object while determining whether inferences are incorrectly generated. Ibid. col. 33 lines 31–64. [a] 3D representation of a pile), a second set of neural network layers that perform object detection (e.g., detecting a pile of material in the single representation),)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Theverapperuma into the teachings of Grossman to provide techniques for volumetric estimation and dimensional estimation of an object. Ibid. col. 1 lines 19–25.
Regarding claim 19, Grossman discloses a method, comprising:
applying, to one or more features of a source image (one or more features of a source image construed as raw images) that depicts a scene using a first set of camera parameters (a first set of camera parameters construed as warped parameters), (Per Fig. 4 at step 410, Grossman discloses a pair of raw images. Then, he applies original warping parameters to determine whether the extent of epipolar misalignment is eminent. Grossman col. 7 line 51 – col. 8 line 4. [a]t 460, no adjustment is needed on the basis of the above epipolar misalignment determination. Note that while no adjustment may be needed on these grounds, additional testing (discussed below) may determine that adjustment of warping parameters is needed for other reasons.) based on a condition view image (a condition view image construed as rectified image(s)) associated with the source image, an epipolar geometric warping to determine a second set of camera parameters (a second set of camera construed as adjusted warped parameters). (Per Fig. 8, Grossman discloses adjusted warping parameters 820 after processing rectified images 350a–350c. Ibid. col. 10 line 61 – col. 11 line 16. This module may receive rectified images 350a . . . c and produce adjusted warping parameters 820.)
However, Grossman fails to specifically disclose generating, using a neural network, a synthetic image representing the one or more features and corresponding to the second set of camera parameters.
In related art, Theverapperuma discloses generating, using a neural network, a synthetic image (a synthetic image construed as a composite image) representing the one or more features (Per Fig. 10, Theverapperuma’s object detection module 1020 discloses a composite image regarding a 3D representation in his CNN model 1022. Theverapperuma col. 30 lines 37–56. The CNN depth model 1022 generates a 3D image based on the first 2D image and the second 2D image.) and corresponding to the second set of camera parameters (A second set of camera parameters construed as a second set of neural network layers in the trained model. See Applicant’s Spec ¶66. [a]nd the second set of camera parameters (e.g., associated with the condition view image 146) to a second neural network.). (Per Fig. 11, Theverappaeruma’s ML model 1120 discloses a second set of neural network layers to detect an object while determining whether inferences are incorrectly generated. Ibid. col. 33 lines 31–64. [a] 3D representation of a pile), a second set of neural network layers that perform object detection (e.g., detecting a pile of material in the single representation),)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Theverapperuma into the teachings of Grossman to provide techniques for volumetric estimation and dimensional estimation of an object. Ibid. col. 1 lines 19–25.
Regarding claim 5, Grossman as modified by Theverapperuma, discloses the one or more processors, wherein the neural network comprises a stable diffusion model. (Per Fig. 2B, Theverapperuma’s perception subsystem 204 discloses a learning model to identify objects whereabouts an autonomous vehicle 120. Theverapperuma col. 12 lines 44–63. A CNN model or other AI/machine learning model built based upon training may then be used in real time to identify and classify objects in the environment of autonomous vehicle 120 based upon new sensor data received from sensors 110.)
Regarding claim 6, Grossman as modified by Theverapperuma, discloses the one or more processors, wherein representations of the one or more features in at least one layer of the neural network are unmodified by the epipoloar geometry warping. (Per Fig. 8, Grossman discloses adjusted warping parameters 820 after processing rectified images 350a–350c. Grossman col. 10 line 61 – col. 11 line 16. This module may receive rectified images 350a . . . c and produce adjusted warping parameters 820.)
Regarding claim 11, Grossman as modified by Theverapperuma, discloses the one or more processors, wherein the one or more processors are comprised in at least one of: a system for generating synthetic data. (Per Fig. 1, Grossman discloses optical systems 110. Grossman col. 3 lines 19–35. [o]ne or more optical systems 110, such as lenses, imagers 120 that may digitize images formed by the lenses,)
Regarding claim 16, it has been rejected in the same manner as claim 5.
Regarding claim 17, it has been rejected in the same manner as claim 6.
Claims 2 and 13 are rejected under 35 U.S.C. § 103 as being unpatentable over Grossman in view of Theverapperuma and further in view of Chandler et al. (U.S. 12,307,603 B2).
Regarding claim 2, Grossman as modified by Theverapperuma, discloses the claimed invention, but fails to specifically disclose the one or more processors wherein the neural network is updated using image pairs, at least one image pair depicting at least one feature of one or more objects in common and including data indicating a relative pose or position of a camera capturing each image of the at least one image pair.
In related art, Chandler discloses the one or more processors wherein the neural network is updated using image pairs, at least one image pair depicting at least one feature of one or more objects in common and including data indicating a relative pose or position of a camera capturing each image of the at least one image pair. (Per Fig. 15A, Chandler discloses a camera pose related to each object. Chandler col. 34 lines 43–67. [a] camera pose relative to each object is estimated and refined in each frame based on the SDF model.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the teachings of Chandler into the teachings of Grossman and Theverapperuma to provide annotated perception inputs fast and efficiently. Ibid. col. 4 lines 8–17.
Regarding claim 13, it has been rejected in the same manner as claim 2.
Allowable Subject Matter
Claims 3–4, 7–10, 14–15, 18 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Benou et al. (U.S. 11,680,801 B2) discloses systems and methods for navigating a host vehicle.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENEDICT LEE whose telephone number is (571)270-0390. The examiner can normally be reached 10:00-16:00 (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R. Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BENEDICT E LEE/Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665
1 See his col. 8 lines 34–48. Under a broadest reasonable interpretation (BRI), Examiner postulates that Grossmann uses current warping parameters in a stereo vision system to calculate corresponding pixel locations if the warping parameters are intact.
2 This 3D representation corresponds to a second set of neural network layers.
Prosecution Insights