DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to the application filed on ----1/26/2026 for application 18/430,377. Claim 1 – 9, 11 – 21 are pending and have been examined.
Claim 1, 6, 9, 11 – 12, 14 – 17 and 19 are amended.
Claim 10 is canceled.
Claim 21 is new.
Claim rejection under 101 are withdrawn in light of the applicants amendment.
Response to Argument
Applicant’s arguments, see page 16 – 20, filed on 1/26/2026, with respect to claim rejection under U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made with the same prior art in view of Lucellent, “How to remove Ultra Wide Angle (pincushion) distortion?”.
Applicant noted in page 18 that “the ‘first segment’ is different than the ‘first sensor representation’ of claim 1. Additionally, Applicant amends independent claim 1 to recite that the ‘first segment ... [is] associated with a first area of the environment’ and the ‘first sensor representation [is] associated with a second area of the environment that is greater than and includes the first area of the environment.’” Examiner notes that the specification does not provide clear definition of each terms rather than exemplary embodiments. Lucellent illustrated images captured by Google Street View with distortions on portions of the captured images (2k4s, “How to remove Ultra Wide Angle (pincushion) distortion”, page 8 – 9), 2k4s suggested to transform and crop the edges off the image to fix the distortion (page 3). In this case, the first sensor representation is mapped to the captured image by wide angle camera which corresponding to a second area of environment and the first segments is mapped to the cropped image from the wide angle camera that corresponding to a first area of environment. The cropped image is corresponding to a smaller and included area before its edge is cropped off. Thus, within BRI, the combination fulfill the limitation. Examiner further notes that Fig. 5 of Brewington illustrated the intervals of picture frames for the “simultaneous location and mapping (‘SLAM’) process to generate a map or 3D model” (0092). Thus the collected picture frames are map data and each picture frame is a sensor representation with a segment that are used for mapping. As point out by applicant in page 18 – 19, Brewington’s system adjusts its frame rate while traveling and thus segment the collected map data into segments.
The remaining arguments are essentially the same as those addressed above and/or below and are unpersuasive for at least the same reasons. Therefore, Examiner is unpersuaded and maintains the corresponding rejections.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1 – 9 and 11 – 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Brewington et al., (hereinafter Brewington), US20200034638 in view of 2k4s, “How to remove Ultra Wide Angle (pincushion) distortion”.
Brewington discloses a street-view mapping application that takes plurality of street pictures and stitches into panoramic views of the environment (0001 - 0002). The system features minimal amount of data storage at the same time provide enough details of the environment by store/discard frames based on amounts of overlap between stored frames (0094 - 0099).
2k4s teaches the image distortion in the mapping application and the solution against the distortion.
Claim 1. Brewington discloses: A method comprising:
obtaining map data representative of a map of an environment (0092, “simultaneous location and mapping ("SLAM") to generate a map or 3D model of targets of interest”; Fig. 5 illustrate the intervals of image taking while driving. The image frames are map data and each of the picture frame corresponds to an area/map of the environment of the vehicle), the map being segmented into a first set of segments, at least a first segment of the first set of segments corresponding to a first area of the environment (0086, “A subset of images (first set of segments) from the images captured by the first image sensor may include images that were captured at or closest to the determined capture locations. The remaining images not in the subset may be dropped before they are transmitted or stored in the memory or may be discarded from the memory in postprocessing.”);
determining a first amount of overlap between a first sensor representation associated with the first segment of the first set of segments and a second sensor representation associated with a second segment of the first set of segments (refer to the mapping above & Fig. 5, & 0067 – 0068, “the circles of set 520 correspond to the capture locations (x, y)”; the first sensor representation and the first segment, in this case, are referring to the image/sensor data of the previous capture location; the second sensor representation and the second segment are referring to images/sensor data near the current capture location);
determining a second amount of overlap between the first sensor representation and a third sensor representation associated with a third segment of the first set of segments (refer to the mapping above, multiple images/sensor data (including third image/sensor data) are taken and the overlaps with previous capture location are considered/compared);
determining, based at least on the first amount of overlap and the second amount of overlap, to remove the third segment from the first set of segments to determine a second set of segments (refer to the mapping above & 0094 – 0099, based on the amount of overlap, the system discard some of the frames/sensor data);
causing, based at least on the second set of segments and using sensor data, at least the first sensor representation to represent a portion of the map that corresponds to the second area of the environment: (refer to the mapping above, the street-view can be constructed with the remaining sensor data. Since the first and second segments have overlap and are both used to construct the map, both first sensor representation and the second sensor representation are used to represent a portion of the map that corresponds to the second area of the environment)
sending, to one or more machines, at least one of the first sensor representation for performing at least one of one or more localization operations, navigation operations, or control operations associated with the one or more machines within the environment (0092, “the vehicle 200 may use simultaneous location and mapping ("SLAM") generate a map or 3D model of targets of interest in real time. The map or model may be used to determine the surfaces that have been captured and the surfaces that still need to be captured … image capture system 100 may be moved to different positions in order to
fill in the gaps and create a more complete model … image capture system is stored on a highly-mobile vehicle, such as an unmanned aerial vehicle ("UAV", e.g., quadcopter)”; i.e., control the operation of machine/UAV using the generated model).
Brewington does not explicitly teach:
the first sensor representation corresponding to a second area of the environment that is greater than and includes the first area of the environment;
2k4s, in the same field of endeavor, explicitly teach:
the first sensor representation corresponding to a second area of the environment that is greater than and includes the first area of the environment (2k4s, page 1, “image from Google Maps … distorted the building is around the edges.” Page 3, “use a keystoning tool to straighten the vertical lines and then crop the image. You will lose some edges.”; i.e., 2k4s teaches preprocessing for the mapping application by transforming the captured image and crop the edges (use only the center portion of the image));
Brewington and 2k4s both teach SLAM mapping application with images from camera and are analogous. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention with a reasonable likelihood of success to further include the preprocessing steps teach by 2k4s in the system of Brewington \to achieve the claimed teaching. One of the ordinary skill in the art would have motivated to make this modification in order reduce the distortion (2k4s, page 1).
Claim 2. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach: determining that the second amount of overlap is greater than the first amount of overlap, wherein the determining to the remove the third segment from the first set of segments is based at least on the second amount of overlap being greater than the first amount of overlap (refer to the mapping above & Brewington, 0094, “the amount of overlap between images may be quite high … thus discard the overlapping image data”; 0099 “the system is to minimize the amount of data capture and storage. The amount of overlap of the collected images and location information above ensures that users have enough information with which to create reconstructions of the geographic area”; Examiner notes that 2k4s teaches to use only the center portion of the captured image for mapping, the overlap between sensor representations are referring to the overlap between the segments that are the cropped center portion of each sensor representations. Examiner further notes that as illustrated in Brewington Fig. 7 and an annotated illustration below, the system captures plurality of picture frames during driving and each of the picture frame corresponds to its area of environment. In order to minimize data storage while making sure no all areas are covered, it is obvious to ordinary skilled in the art to keep the less overlapped segments and discard the heavily overlapped segments. In this case, the entire third area (third segment) would be dropped because its area is covered by/redundant to both sides of segments (first and second)).
PNG
media_image1.png
969
1464
media_image1.png
Greyscale
Claim 3. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach: determining that the second amount of overlap is equal to or greater than a threshold amount of overlap, wherein the determining to remove the third segment from the first set of segments is further based at least on the second amount of overlap being equal to or greater than the threshold amount of overlap (Brewington, 0076, “the image capture system 100 may determine a rate of image capture such that the amount of overlap between two sequential images of a target of interest exceeds a threshold.”).
Claim 4. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach: determining that a distance between a center of the first sensor representation and a center of the second sensor representation is less than or equal to a distance threshold, wherein the determining to remove the third segment from the first set of segments is further based at least on the distance being less than or equal to the distance threshold (Brewington, 0076, “based on an angle of view of the first image sensor of the one or more image sensors 120 and an average distance of the target of interest from the first image sensor, one or more capture locations may be determined that would allow an at least 50% overlap between images of the target of interest”; the distance between center represents the overlap of subsequent image; examiner notes that the center of the sensor representation and the segments that are cropped for mapping purpose should be same because the cropping remove all edges that are away from the center and distorted).
Claim 5. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach: determining that the third sensor representation further includes an overlap with at least a fourth sensor representation associated with a fourth segment of the first set of segments and a fifth sensor representation associated with a fifth segment of the first set of segments, wherein the determining to remove the third segment from the first set of segments is further based at least on the third sensor representation further including the overlap with the fourth sensor representation and the fifth sensor representation (refer to the mapping in Claim 1 & Brewington, 0086, The system collects series of images/sensor data and the post process compares these sensor data to determine which ones will be stored/discarded. Fig. 5 illustrates the driving path which also represents the image collection sequences. In order the reduce the storage, the images that are challenging/comparing to the prior image to represent an target of interest will also be challenged/compared to the following image; thus a fourth, fifth segments/images/sensor data).
Claim 6. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach: determining to remove the third sensor representation occurs during a first iteration, and wherein the method further comprises: determining a third amount of overlap between the first sensor representation associated with the first segment of the second set of segments and the second sensor representation associated with the second segment of the second set of segments; determining a fourth amount of overlap between the first sensor representation and a fourth sensor representation associated with a fourth segment of the second set of segments; and determining, during a second iteration and based at least on the third amount of overlap and the fourth amount of overlap, to remove the fourth segment from the second set of segments to generate a third set of segments, wherein the causing the first sensor representation to represent the portion of the map is based at least on the third set of images (refer to the mapping in Claim 1, 5, & Brewington, 0093 – 0099, the system collect a plurality of images/sensor data around the location of interest and store/discard the plurality of images/sensor data based on the post processing steps until (iteratively) the system find a good representation that use minimal amount of data having enough information to create reconstruction of a target of interest; fig. 5 & 0003, “identify one or more targets of interest”; 0006, “capturing the first target of interest … capturing the second target of interest”; i.e., plurality of targets of interest thus plurality of iterations).
Claim 7. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach:
determining a third amount of overlap between a fourth sensor representation associated with a fourth segment of the first set of segments and a fifth sensor representation associated with a fifth segment of the first set of segments, wherein the determining to remove the third segment from the first set of segments is further based at least on the third amount of overlap (refer to the mapping in Claim 1 & 5, a plurality of images are collected and compared around the target of interest and are stored/discarded based on the overlap. Thus, a fourth, fifth segments/images/sensor data).
Claim 8. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach:
the first set of segments includes at least a first identifier associated with the first segment, a second identifier associated with the second segment, and a third identifier associated with a third segment; and to remove the third segment from the first set of segments comprises removing the third identifier from the first set of segments (Brewington, 0067, “images or location data may be stored in the memory 230 or the storage system 350 and retrieved by the one or more computing devices, such as the computing devices 110, as needed.”; the identifier of data in the memory is inherent (for example pointers). Once the image/sensor data is determined discarded, the memory will be collected and pointer deleted/removed).
Claim 9. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach:
the sensor data comprises RADAR data; and the causing the first sensor representation to represent the portion of the map comprises causing the first sensor representation to be generated using a portion of the RADAR data that is associated with the second area of the environment corresponding to the portion of the map (Brewington, 0040, “the image capture system may include, for example, radar”; Brewington 0042, “image stitched together from multiple images”; 2k4s, “multiple images stitched together”; i.e., the overlapping portion of the sensor representations/segments are stitched together. Since the first sensor representation and second sensor representation has overlap (where the stitching happens), first sensor representation is used to generate portion of the map data).
Regarding Claim 11. Claim 11 is a system claim corresponding to Claim 1 but broader. The first relationship, the second relationship, the first representation dimension and the second representation dimension is referring to the dimensional overlap described in Claim 1. The a portion of limitations corresponding to Claim 1. Brewington discloses a system to perform the task as illustrated in figure 2 – 4. The first, second and third representation dimensions, the first, second and third portion of a map in this case corresponds to the plurality of images/sensor data and their image/sensor data represented dimension of the environment. The first and second relationship correspond to the first and second overlap of Claim 1. A first portion of the map and fourth portion of the map are referring to the first area of the environment and the second area of the environment.
Thus, Claim 11 is rejected with same reason as Claim 1.
Regarding 12. Based on Claim 11, Claim 12 is a system claim that describes the corresponding limitation of Claim 1. As explained in Claim 11, the first, second and third representation dimensions are corresponds to first, second and third segments of Claim 1 which are map to the plurality of sensor data of Brewington. The system of Brewington selectively choose frames of sensor data to represents the map based on the overlap. Thus Claim 12 are rejected with the same reason as Claim 1.
Claim 13. Claim 13 is the corresponding system claim of Claim 2. Claim 13 is rejected with same reason.
Claim 14. Brewington and 2k4s combination teaches all the limitation in Claim 11, the combination further teach: the one or more first relationships include at least a first distance between a first point associated with the first representation dimensions and a second point associated with the second representation dimensions; the one or more second relationships include at least a second distance between the first point associated with the first representation dimensions and a third point associated with the third representation dimensions; the one or more processors are further to determine that at least one of the first distance or the second distance is less than or equal to a distance threshold; and the generation of the sensor representation is caused based at least on the at least one of the first distance or the second distance being less than or equal to the distance threshold (the first, second third points are corresponding to the center of the first, second and third sensor representation in Claim 4. Claim 14 are rejected with same reason. Claim 14 is rejected with same reason).
Claim 15. Brewington and 2k4s combination teaches all the limitation in Claim 11, the combination further teach: determine that at least one of the one or more first relationships or the one or more second relationships satisfy one or more thresholds, wherein the generation of the sensor representation is further caused based at least on the at least one of the one or more first relationships or the one or more second relationships satisfying the one or more thresholds (refer to the mapping in Claim 3, the relationship are mapping to the overlap and the thresholds are mapping to the amount of overlap in Claim 3. Claim 15 is rejected with same reason).
Claim 16. Brewington and 2k4s combination teaches all the limitation in Claim 11, the combination further teach: determine, during a first iteration and based at least on the one or more first relationships and the one or more second relationships, to refrain from causing generation of a second sensor representation to represent a fifth portion of the map; determine one or more third relationships between the first representation dimensions and the second representation dimensions; determine one or more fourth relationships between the first representation dimensions and fourth representation dimensions associated with a fourth segment of the map; and determine, during a second iteration and based at least on the one or more third relationships and the one or more fourth relationships, to refrain from causing generation of a third sensor representation to represent a sixth portion of the map (refer to the mapping in Claim 6, the relationship is corresponding to the overlap of Claim 6. Brewington teaches taking sensor representation of a plurality of targets of interest and each target of interest can be processed in iteration. Claim 16 is rejected with same reason).
Claim 17. Brewington and 2k4s combination teaches all the limitation in Claim 11, the combination further teach: the sensor data comprises RADAR data; and the causation of the generation of the sensor representation to represent the fourth portion of the map comprises: determining a portion of the RADAR data that is associated with the fourth portion of the map; and causing the generation of the sensor representation using the portion of the RADAR data (refer to the mapping in Claim 9. Brewington teaches that the sensor can include radar. Claim 17 is rejected with same reason).
Claim 18. Brewington and 2k4s combination teaches all the limitation in Claim 11, the combination further teach: the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative Al operations; a system for performing operations using one or more large language models (LLMs); a system for performing one or more conversational Al operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (refer to mapping of Claim 11 & Claim 1. Brewington teaches that the system can be unmanned aerial vehicle (control system for an autonomous or semi-autonomous machine)).
Regarding Claim 19. Claim 19 is corresponding one or more processors claim of Claim 1 with broader limitation. Brewington teaches computing device with processors as illustrated in Fig. 2. Brewington further teach “computing devices 110 or 210 may set the rate of capture of each of the one or more sensors or may cause the one or more sensors to capture data at a determined trigger location” (0091), and thus segments its image frames/map segments in the area. Thus, Claim 19 is rejected with same reason.
Regarding Claim 20, depending on Claim 19, recites same limitation as Claim 18. Claim 20 is rejected with same reason.
Regarding Claim 21. Brewington and 2k4s combination teaches all the limitation in Claim 1, the combination further teach: the third segment is associated with a third area of the environment that borders the first area of the environment (refer to the mapping in Claim 1, Brewington Fig. 7 & illustration in Claim 2, the third segment is next/border to the first segment); and
the third sensor representation is associated with a fourth area of the environment that includes at least a portion of the first area of the environment and at least a portion of the third area of the environment (refer to the mapping above and Claim 1, in this case, the third segment is the cropped map data of the third sensor representation (that covers fourth area of environment), thus, the fourth area of environment includes/covers a portion of first area of environment and third area of environment).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure: Montero. JR. et al., US20250166311, which teaches the method and system that utilize takes recorded video/sensor data to generate virtual view of environment in a sequence of iterative locations.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIEN MING CHOU whose telephone number is (571)272-9354. The examiner can normally be reached Monday- Friday 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HITESH PATEL can be reached on 571-270-5442. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIEN MING CHOU/Examiner, Art Unit 3666
/Hitesh Patel/Supervisory Patent Examiner, Art Unit 3667
6/9/26
Prosecution Insights