DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is in response to Applicant’s amendment/response filed on 30 January 2026, which has been entered and made of record.
Response to Arguments
Applicant’s arguments have been fully considered but they are moot in view of the new grounds of rejection presented in this Office Action.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-6, and 8-11 are rejected under 35 U.S.C. 103 as being unpatentable over Suhail et al. (“Light Field Neural Rendering”; hereinafter “Suhail”) in view of Davis et al. (US 2008/0247635; hereinafter “Davis”), and further in view of Adamkiewicz et al. (“Vision-Only Robot Navigation in a Neural Radiance World”; hereinafter “Adamkiewicz”).
Regarding claim 1, Suhail discloses A method comprising: sampling a set of samples along a ray intersecting a representation of at least one object (“sample a sequence of P points … along the ray,” pg. 8271, sec. 3.2; Fig. 2 illustrates the ray intersecting a representation of an object); using at least one neural network (Fig. 2 illustrates multiple Transformer and MLP neural networks) to obtain a set of feature values based at least in part on the set of samples (“sample a sequence of P points … along the ray … features associated to the epipolar points and target ray,” pg. 8271, sec. 3.2); using the set of feature values to obtain at least one pooled feature value; using at least one pooled feature value (“aggregating features,” pg. 8271, sec. 3.2) to obtain a color value (“predicts the target ray color by aggregating features,” pg. 8271, sec. 3.2).
Suhail does not disclose the representation being a three-dimensional representation of the object.
In the same art of image analysis and modeling of a real-world object, Davis teaches sampling a set of samples along a ray intersecting a three-dimensional representation of at least one object (“The projected ray may pass through several surfaces of the 3D model, defining an intersection point for each surface through which the projected ray passes,” para. 35; “intersection points of each projected ray with the 3D model mesh should be found in the S samples,” para. 38; “The 3D model may thus be conceptualized as a virtual 3D image (virtual object) of a corresponding real object … The image files may comprise high resolution images generated while scanning the object to be inspected for purposes of creating the corresponding 3D model,” paras. 18-19).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Davis to Suhail. The motivation would have been to “reveal features or other characteristics of the real object that may be of interest” (Davis, para. 18).
The combination of Suhail and Davis does not disclose causing a device to move based at least in part on a path of motion determined based at least in part on the color value.
In the same art of neural rendering, Adamkiewicz teaches causing a device to move based at least in part on a path of motion determined based at least in part on the color value (“navigating a robot through a 3D environment represented as a NeRF using only an onboard RGB camera for localization … a trajectory optimization algorithm that avoids collisions with high-density regions in the NeRF … an optimization based filtering method to estimate 6DoF pose and velocities for the robot in the NeRF given only an onboard RGB camera,” abstract; Fig. 2 illustrates using a color output of a neural renderer to update a trajectory).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Adamkiewicz to the combination of Suhail and Davis. The motivation would have been “Planning and executing a trajectory with onboard sensors is a fundamental building block of many robotic applications” (Adamkiewicz, pg. 1, sec. I, para. 1).
Regarding claim 3, the combination of Suhail, Davis, and Adamkiewicz renders obvious encoding the set of samples with positional information indicating a position of each of the set of samples with respect to at least one other of the set of samples, the at least one neural network using the positional information to obtain the set of feature values (“positional encoding … We aggregate the P outputs corresponding to the epipolar points … to obtain the reference view features,” Suhail, pg. 8272, sec. 3.3.1).
Regarding claim 4, the combination of Suhail, Davis, and Adamkiewicz renders obvious wherein the at least one neural network comprises a transformer encoder to obtain the set of feature values (“Epipolar feature transformer,” Suhail, pg. 8272, sec. 3.3.1)
Regarding claim 5, the combination of Suhail, Davis, and Adamkiewicz renders obvious calculating the at least one pooled feature value based at least in part on the set of feature values (“computes a feature representation per reference view by aggregating features associated to the epipolar points and target ray,” Suhail, pg. 8271, sec. 3.2).
Regarding claim 6, the combination of Suhail, Davis, and Adamkiewicz renders obvious wherein the set of feature values comprises a plurality of values for a plurality of features, the at least one pooled feature value comprises a corresponding pooled feature value for each of the plurality of features, at least a portion of the plurality of values are associated with each of the plurality of features (Suhail, Fig. 2 illustrates these properties of the features and values), and for each of the plurality of features, the corresponding pooled feature value is calculated as a maximum or an average of those of the plurality of values in the portion associated with the feature (“The aggregation is a weighted average,” Suhail, pg. 8272, sec. 3.3.1).
Regarding claim 8, the combination of Suhail, Davis, and Adamkiewicz renders obvious wherein the at least one neural network comprises a multilayer perceptron that uses the at least one pooled feature value to obtain the color value (Suhail, Fig. 2 illustrates the feature aggregation used as input to an MLP to predict color).
Regarding claim 9, the combination of Suhail, Davis, and Adamkiewicz renders obvious wherein the at least one neural network uses a photometric loss function to obtain the color value (“During training, we minimize the L2 loss between the observed and predicted colors,” Suhail, pg. 8273, sec. 3.4; “a NeRF-based photometric loss,” Adamkiewicz, pg. 2, col. 1; see claim 1 for motivation to combine).
Regarding claim 10, the combination of Suhail, Davis, and Adamkiewicz renders obvious determining the path of motion based at least in part on the color value (Adamkiewicz, Fig. 2 illustrates using a color output of a neural renderer to update a trajectory; see claim 1 for motivation to combine).
Regarding claim 11, the combination of Suhail, Davis, and Adamkiewicz renders obvious casting the ray through the three-dimensional representation (The projected ray may pass through several surfaces of the 3D model, defining an intersection point for each surface through which the projected ray passes,” Davis, para. 35; see claim 1 for motivation to combine).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Suhail, Davis, and Adamkiewicz, and further in view of Mildenhall et al. (“NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis”; hereinafter “Mildenhall”).
Regarding claim 2, the combination of Suhail, Davis, and Adamkiewicz does not disclose dividing a region into subregions, and obtaining a sample of the set of samples along the ray from each of the subregions.
In the same art of neural rendering, Mildenhall teaches dividing a region into subregions, and obtaining a sample of the set of samples along the ray from each of the subregions (“near and far bounds tn and tf … we use a stratified sampling approach where we partition [tn, tf] into N evenly spaced bins and then draw one sample uniformly at random from within each bin,” pg. 101, sec. 4).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Mildenhall to the combination of Suhail, Davis, and Adamkiewicz. The motivation would have been “stratified sampling enables us to represent a continuous scene representation” (Mildenhall, pg. 101, sec. 4).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Suhail, Davis, and Adamkiewicz, and further in view of Hori et al. (US 2024/0046085; hereinafter “Hori”).
Regarding claim 7, the combination of Suhail, Davis, and Adamkiewicz renders obvious wherein the at least one neural network comprises a transformer encoder (“Epipolar feature transformer,” Suhail, pg. 8272, sec. 3.3.1) comprising at least one layer comprising self-attention functionality (“self-attention transformer,” Suhail, pg. 8272, sec. 3.3.1).
The combination of Suhail, Davis, and Adamkiewicz does not disclose that the self-attention functionality determines a set of dependency metrics based at least in part on a set of parameter values associated with the set of samples and feed forward functionality that outputs one or more feature values based at least in part on the set of dependency metrics.
In the same art of extracting visual features using machine learning, Hori teaches self-attention functionality that determines a set of dependency metrics based at least in part on a set of parameter values associated with the set of samples and feed forward functionality that outputs one or more feature values based at least in part on the set of dependency metrics (“The self-attention layer 410 [of Fig. 4] extracts temporal dependency … the feed-forward layers 412 are applied in a point-wise manner. The encoded representations for audio and visual features are obtained,” para. 73).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Hori to the combination of Suhail, Davis, and Adamkiewicz. The motivation would have been to optimize the feature extraction.
Claims 12, 14-20, 24, 25, 27, and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Suhail in view of Davis.
Regarding claim 12, Suhail discloses A system comprising: at least one processor; and memory storing instructions that when executed by the at least one processor cause the system to (these are considered inherent aspects of a computer-based neural rendering architecture): perform one or more transformer models (“transformer-based model,” abstract) to obtain a set of feature values based at least in part on a set of locations along a ray intersecting a representation of an object (“computes a feature representation per reference view by aggregating features associated to the epipolar points and target ray,” pg. 8271, sec. 3.2); perform one or more machine learning processes to obtain a color value based at least in part on the set of feature values (“predicts the target ray color by aggregating features associated to each reference view,” pg. 8271, sec. 3.2); and generate a view of the object using the color value (“synthesize novel views of a scene,” pg. 8271, sec. 3).
Suhail does not disclose the representation being a three-dimensional representation of the object.
In the same art of image analysis and modeling of a real-world object, Davis teaches a set of locations along a ray intersecting a three-dimensional representation of an object (“The projected ray may pass through several surfaces of the 3D model, defining an intersection point for each surface through which the projected ray passes,” para. 35, “intersection points of each projected ray with the 3D model mesh should be found in the S samples,” para. 38; “The 3D model may thus be conceptualized as a virtual 3D image (virtual object) of a corresponding real object … The image files may comprise high resolution images generated while scanning the object to be inspected for purposes of creating the corresponding 3D model,” paras. 18-19).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Davis to Suhail. The motivation would have been to “reveal features or other characteristics of the real object that may be of interest” (Davis, para. 18).
Regarding claim 14, the combination of Suhail and Davis renders obvious wherein the one or more transformer models obtain the set of feature values using positional encodings obtained based at least in part on the set of locations (“positional encoding … We aggregate the P outputs corresponding to the epipolar points … to obtain the reference view features,” Suhail, pg. 8272, sec. 3.3.1).
Regarding claim 15, the combination of Suhail and Davis renders obvious wherein the one or more transformer models comprises a transformer encoder (“Epipolar feature transformer,” Suhail, pg. 8272, sec. 3.3.1).
Regarding claim 16, the combination of Suhail and Davis renders obvious aggregate at least a portion of the set of feature values to obtain an aggregated feature value, the one or more machine learning processes to obtain the color value based at least in part on the aggregated feature value (“predicts the target ray color by aggregating features,” Suhail, pg. 8271, sec. 3.2).
Regarding claim 17, the combination of Suhail and Davis renders obvious wherein each of the set of feature values is associated with a feature in a set of features (see Suhail, Fig. 2), and the portion of the set of feature values are aggregated by at least one of selecting a maximum one of the set of feature values for each feature within the set of features or calculating an average of those of the set of feature values associated with each feature within the set of features (“The aggregation is a weighted average,” Suhail, pg. 8272, sec. 3.3.1).
Regarding claim 18, the combination of Suhail and Davis does not specifically recite wherein the one or more transformer models comprises a transformer encoder usable for natural language processing.
The Examiner previously took Official Notice that both the concepts and the advantages of using a transformer encoder for natural language processing were well known and expected in the art before the effective filing date of the claimed invention. Since Applicant did not traverse the Official Notice, it is now taken as Applicant Admitted Prior Art. It would have been obvious before the effective filing date of the claimed invention to use a transformer encoder usable for natural language processing in the combination of Suhail and Davis in order to increase accuracy and efficiency of the machine learning model.
Regarding claim 19, the combination of Suhail and Davis renders obvious wherein the one or more machine learning processes comprise a multilayer perceptron to obtain the color value (Suhail, Fig. 2 illustrates the feature aggregation used as input to an MLP to predict color).
Regarding claim 20, the combination of Suhail and Davis renders obvious wherein the one or more machine learning processes uses a photometric loss function when obtaining the color value (“During training, we minimize the L2 loss between the observed and predicted colors,” Suhail, pg. 8273, sec. 3.4).
Regarding claim 24, it is rejected using the same citations and rationales described in the rejection of claim 12.
Regarding claim 25, the combination of Suhail and Davis renders obvious cast the ray through the three-dimensional representation of the obstacle and between a virtual image capture device and a focal point of the virtual image capture device (see “Target Image” of Suhail, Fig. 2; “Given a target ray to render,” Suhail, Fig. 2 caption; “The projected ray may pass through several surfaces of the 3D model, defining an intersection point for each surface through which the projected ray passes,” Davis, para. 35; see claim 12 for motivation to combine).
Regarding claim 27, the combination of Suhail and Davis renders obvious wherein a multilayer perceptron is to obtain the at least one color value (Suhail, Fig. 2 illustrates the feature aggregation used as input to an MLP to predict color), and the at least one transformer-based machine learning model comprises a transformer encoder to obtain the set of feature values (“Epipolar feature transformer,” Suhail, pg. 8272, sec. 3.3.1).
Regarding claim 28, the combination of Suhail and Davis renders obvious use a photometric loss function to supervise performing the at least one transformer-based machine learning model and determining the at least one color value (“During training, we minimize the L2 loss between the observed and predicted colors,” Suhail, pg. 8273, sec. 3.4).
Claims 13 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Suhail and Davis, and further in view of Mildenhall.
Regarding claim 13, the combination of Suhail and Davis discloses obtaining the set of locations (“sample a sequence of P points … along the ray,” Suhail, pg. 8271, sec. 3.2).
The combination of Suhail and Davis does not disclose dividing a portion of the ray within bounds of the three-dimensional representation of the object into regions and obtain a sample from within each region.
In the same art of neural rendering, Mildenhall teaches dividing a portion of the ray within bounds of the three-dimensional representation of the object into regions and obtain a sample from within each region (“near and far bounds tn and tf … we use a stratified sampling approach where we partition [tn, tf] into N evenly spaced bins and then draw one sample uniformly at random from within each bin,” pg. 101, sec. 4).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Mildenhall to the combination of Suhail and Davis. The motivation would have been “stratified sampling enables us to represent a continuous scene representation” (Mildenhall, pg. 101, sec. 4).
Regarding claim 26, it is rejected using the same citations and rationales described in the rejection of claim 13.
Claims 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Suhail and Davis, and further in view of Adamkiewicz.
Regarding claim 21, the combination of Suhail and Davis does not disclose a device, wherein the instructions, when executed by the at least one processor, cause the at least one processor to determine a path of motion based at least in part on the color value, and instruct the device to move based at least in part on the path of motion.
In the same art of neural rendering, Adamkiewicz teaches a device, wherein the instructions, when executed by the at least one processor, cause the at least one processor to determine a path of motion based at least in part on the color value, and instruct the device to move based at least in part on the path of motion (“navigating a robot through a 3D environment represented as a NeRF using only an onboard RGB camera for localization … a trajectory optimization algorithm that avoids collisions with high-density regions in the NeRF … an optimization based filtering method to estimate 6DoF pose and velocities for the robot in the NeRF given only an onboard RGB camera,” abstract; Fig. 2 illustrates using a color output of a neural renderer to update a trajectory).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Adamkiewicz to the combination of Suhail and Davis. The motivation would have been “Planning and executing a trajectory with onboard sensors is a fundamental building block of many robotic applications” (Adamkiewicz, pg. 1, sec. I, para. 1).
Regarding claim 22, the combination of Suhail, Davis, and Adamkiewicz renders obvious wherein the device is an autonomous machine or a semi-autonomous machine (“autonomous driving or drone flight,” Adamkiewicz, pg. 1, sec. I; see claim 21 for motivation to combine).
Regarding claim 23, the combination of Suhail, Davis, and Adamkiewicz renders obvious wherein the device is an autonomous vehicle (“autonomous driving or drone flight,” Adamkiewicz, pg. 1, sec. I; see claim 21 for motivation to combine), and the instructions, when executed by the at least one processor, cause the at least one processor to: obtain image data captured by at least one image capture device associated with the autonomous vehicle (“Camera Images” of Adamkiewicz, Fig. 2; see claim 21 for motivation to combine), generate the three-dimensional representation of the object based on the image data (“The 3D model may thus be conceptualized as a virtual 3D image (virtual object) of a corresponding real object … The image files may comprise high resolution images generated while scanning the object to be inspected for purposes of creating the corresponding 3D model,” Davis, paras. 18-19; see claim 12 for motivation to combine); and cast the ray through the three-dimensional representation of the object (“The projected ray may pass through several surfaces of the 3D model, defining an intersection point for each surface through which the projected ray passes,” Davis, para. 35; see claim 12 for motivation to combine).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan McCulley whose telephone number is (571)270-3754. The examiner can normally be reached Monday through Friday, 8:00am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN MCCULLEY/Primary Examiner, Art Unit 2611
Prosecution Insights