Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 7, and 11-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lawler et al. (U.S. PGPUB 20170213383) in view of Aizawa (U.S. PGPUB 20220321855).
With respect to claim 1, Lawler et al. disclose a method comprising: obtaining one or more user inputs to an oblique image visualization system, wherein the oblique image visualization system is associated with a plurality of non-composite oblique aerial images of a geographic area (paragraph 38, In regard to the operations on a user computer, reference is now made to FIG. 4. FIG. 4 is a flow diagram illustrating an exemplary routine 400 for displaying geographic data on an image taken at an oblique angle. Beginning at block 402, an image is requested from an online image source, such as online source 112 of FIG. 1);
determining, based on the one or more user inputs, view perspective information (paragraph 34, At block 212, the metadata for the image, comprising the approximate elevations, the depth bitmap, and a projection matrix are organized as the generated metadata) indicative of a user-requested view of a scene or area within the geographic area (paragraph 38, At block 404, the image is obtained from the online source. Additionally, as set forth in block 406, metadata corresponding to the obtained image is also obtained); and
outputting the oblique image for display by the oblique image visualization system in response to the user-requested view (paragraph 39, At block 410, the geographic data is mapped/displayed on the image according to the obtained metadata corresponding to the image. Mapping (and displaying) geographic data on an image taken at an oblique angle according to metadata corresponding to the image is set forth in more detail in regard to FIG. 5, paragraph 42, displaying/locating the current element at its location on the image is made according to the projection matrix based on the coordinates of the element and the elevation). However, Lawler et al. do not expressly disclose comparing one or more parameters of the view perspective information to a corresponding one or more parameters associated with each oblique image of the plurality of non-composite oblique aerial images; selecting a best match oblique image from the plurality of non-composite oblique aerial images based on the comparison; and
outputting the best match oblique image.
Aizawa, who also deals with rendering an image, disclose a method for comparing one or more parameters of the view perspective information to a corresponding one or more parameters associated with each oblique image of the plurality of non-composite oblique aerial images (paragraph 57, comparisons may be made between parameters for both the position and the image capturing direction of each of the image capture apparatuses identified in S502 and parameters for both the position of the specified viewpoint and the eye direction from the specified viewpoint);
selecting a best match oblique image from the plurality of non-composite oblique aerial images based on the comparison (paragraph 57, with differences for both parameters taken into consideration, an image capture apparatus with the smallest differences may be selected, paragraph 60, In S510, from the captured image managing unit 206, the selected image obtaining unit 213 obtains the captured image captured by the image capture apparatus selected by the image capture apparatus selecting unit 212); and
outputting the best match oblique image (paragraph 61, In S511, the captured image display controlling unit 214 displays the image obtained in S510).
Lawler et al. and Aizawa are in the same field of endeavor, namely computer graphics.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method of comparing one or more parameters of the view perspective information to a corresponding one or more parameters associated with each oblique image of the plurality of non-composite oblique aerial images; selecting a best match oblique image from the plurality of non-composite oblique aerial images based on the comparison; and outputting the best match oblique image, as taught by Aizawa, to the Lawler et al. system, because the image capture apparatus may be selected based on at least one of the position of the specified viewpoint and the eye direction from the specified viewpoint (paragraph 57 of Aizawa) and this captures images of a scene that the user wants to check, from a viewpoint which is the same as or closer to a desired viewpoint (paragraph 61 of Aizawa).
With respect to claim 2, Lawler et al. as modified by Aizawa disclose the method of claim 1, further comprising: determining a difference between values of the one or more parameters of the view perspective information and respective values of the corresponding one or more parameters associated with each oblique image (Aizawa: paragraph 57, comparisons may be made between parameters for both the position and the image capturing direction of each of the image capture apparatuses identified in S502 and parameters for both the position of the specified viewpoint and the eye direction from the specified viewpoint. And with differences for both parameters taken into consideration); see rationale for rejection of claim 1.
With respect to claim 3, Lawler et al. as modified by Aizawa disclose the method of claim 2, wherein the best match oblique image is selected based on having a minimum difference from the values of the one or more parameters of the view perspective information (Aizawa: paragraph 57, an image capture apparatus with the smallest differences may be selected); see rationale for rejection of claim 1.
With respect to claim 4, Lawler et al. as modified by Aizawa disclose the method of claim 1, wherein the geographic area is a geographic area of interest identified from one or more user inputs (paragraph 63, In S701, the camera path setting unit 209 obtains user instructions regarding a camera path, paragraph 69, In S702, the camera path setting unit 209 determines whether input values indicating the user instructions obtained in S701 include an instruction to adjust parameters indicative of orientation, paragraph 70, In S703, the camera path setting unit 209 derives a region to trim part of the image capture range of each of the image capture apparatuses identified in S502. An image showing a cropped region is referred to as a trimmed image), and wherein the view perspective information includes at least a center point coordinate of the user-requested view of the scene or area within the geographic area of interest (paragraph 72, Positions 701a to 701c are the positions of the judgment target 601 on the images 801a to 801c, respectively, paragraph 77, A point 902 is the center of the image 911. A point 903 is the image center of the trimmed image 912 trimmed from the image 911). It would have been obvious to apply the teachings of Aizawa because this makes it easier for the user to check the virtual viewpoint images. Thus, orientation adjustments can reduce the load on the user in checking the virtual viewpoint images (paragraph 80 of Aizawa).
With respect to claim 7, Lawler et al. as modified by Aizawa disclose the method of claim 1, wherein the view perspective information is indicative of one or more of: a center point coordinate of the user-requested view; an oblique view angle of the user-requested view (Lawler et al.: paragraph 33, The projection matrix provides the basis for locating a geographic coordinate within an image taken at an oblique angle. Indeed, the projection matrix comprises translation information and/or formulae that enable a location to be plotted on an image from an oblique angle, give the latitude, longitude, and elevation of that point (geographic location). While based on the set of information, the projection matrix may also include the geographic coordinates, elevation, orientation, and lens properties of the camera lens at the moment that the image was captures); a heading or orientation angle of the user-requested view; or a zoom level of the user-requested view.
With respect to claim 11, Lawler et al. as modified by Aizawa disclose the method of claim 1, wherein outputting the best match oblique image for display includes cropping the best match oblique image data to match the user-requested view of the scene or area within the geographic area (Aizawa: paragraph 70, In S703, the camera path setting unit 209 derives a region to trim part of the image capture range of each of the image capture apparatuses identified in S502. An image showing a cropped region is referred to as a trimmed image). It would have been obvious to apply the teachings of Aizawa because orientation adjustments can reduce the load on the user in checking the virtual viewpoint images (paragraph 80 of Aizawa).
With respect to claim 12, Lawler et al. as modified by Aizawa disclose the method of claim 11, wherein the best match oblique image data is cropped based on one or more of a configured zoom level or a configured center point coordinate indicated in the view perspective information for the user-requested view (Aizawa: paragraph 80, In a case where a region to crop is determined so that the judgment target may be projected onto the center of the trimmed image, the virtual viewpoint images representing the views from the respective adjusted viewpoints are generated such that the judgment target is near the center of the images); see rationale for rejection of claim 11.
With respect to claim 13, Lawler et al. as modified by Aizawa disclose a system (Lawler et al.: paragraph 61, FIG. 8 is a block diagram illustrating an exemplary user computing device suitable for displaying geographic data on an image taken from an oblique angle) comprising: one or more processors; and one or more computer-readable storage media having computer-readable instructions stored thereon (Lawler et al.: paragraph 61, the exemplary user computing device 800 includes one or more processors (or processing units), such as processor 802, and a memory 804), wherein the computer-readable instructions, when executed by the one or more processors, cause the one or more processors to execute the method of claim 1; see rationale for rejection of claim 1.
With respect to claim 14, Lawler et al. as modified by Aizawa disclose the system of claim 13, wherein the computer-readable instructions further cause the one or more processors to execute the method of claims 2-3; see rationale for rejection of claims 2-3.
Claim(s) 5-6, 8-10, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lawler et al. (U.S. PGPUB 20170213383) in view of Aizawa (U.S. PGPUB 20220321855) and further in view of Balfour (U.S. PGPUB 20070024612).
With respect to claim 5, Lawler et al. as modified by Aizawa disclose the method of claim 4. However, Lawler et al. as modified by Aizawa do not expressly disclose each oblique image of the plurality of non-composite oblique aerial images is associated with a respective image frame center coordinate; and the best match oblique image is selected based on having a smallest distance between the center point coordinate of the user-requested view and the respective image frame center coordinate.
Balfour, who also deals with rendering an image, discloses a method wherein each oblique image of the plurality of non-composite oblique aerial images is associated with a respective image frame center coordinate (paragraph 44, the image center point is pre-calculated offset into the lower half of the image. The geo-polys (22) data is a compilation of all this data for all images); and the best match oblique image is selected based on having a smallest distance between the center point coordinate of the user-requested view and the respective image frame center coordinate (paragraph 47, The other 80% of the final heuristic score is preferably based on the delta distance that the center point of each image (contained in the geo-polys (22) data) in the immediate neighbor list (21) is from the current surface reference point (18). If the highest scoring image is still the currently displayed oblique image, the current oblique image (24) remains unchanged for this 3D/4D viewer (5) update cycle).
Lawler et al., Aizawa, and Balfour are in the same field of endeavor, namely computer graphics.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method wherein each oblique image of the plurality of non-composite oblique aerial images is associated with a respective image frame center coordinate; and the best match oblique image is selected based on having a smallest distance between the center point coordinate of the user-requested view and the respective image frame center coordinate, as taught by Balfour, to the Lawler et al. as modified by Aizawa system, because this would provide a natural and easy interface for the user to effectively and efficiently access and analyze oblique imagery from a large image warehouse (paragraph 8 of Balfour) and provide the ability to automatically select the current oblique image of interest from a large image warehouse, based on the user's viewpoint within a virtual 3D/4D scene, and display that image geo-registered in the virtual 3D/4D scene (paragraph 9 of Balfour).
With respect to claim 6, Lawler et al. as modified by Aizawa and Balfour disclose the method of claim 5, wherein the center point coordinate of the user-requested view and the respective image frame center coordinate for each oblique image are defined using a same common coordinate system (Balfour: paragraph 47, The other 80% of the final heuristic score is preferably based on the delta distance that the center point of each image (contained in the geo-polys (22) data) in the immediate neighbor list (21) is from the current surface reference point (18)); see rationale for rejection of claim 5. As shown in the Fig. 2, the difference is calculated in the same coordinate system relative to surface reference point (18).
With respect to claim 8, Lawler et al. as modified by Aizawa and Balfour disclose the method of claim 7, comprising: filtering the plurality of non-composite oblique aerial images to obtain a filtered subset of oblique images, wherein the filtered subset of oblique images each have corresponding parameter values matching with one or more of the oblique view angle and the heading or orientation angle of the user-requested view (Balfour: paragraph 47, a preferred embodiment ranks all images in the immediate neighbor list (21) of the currently displayed oblique image by the images' heading deviance from the current viewpoint (17), preferably counted as 20% of the final heuristic score, and preferably eliminating any neighbor image with a heading deviance greater than 30 degrees). It would have been obvious to apply the teachings of Balfour; see rationale for rejection of claim 5.
With respect to claim 9, Lawler et al. as modified by Aizawa and Balfour disclose the method of claim 8, wherein the best match oblique image is determined based on a calculated straight line distance between the center point coordinate of the user-requested view and a respective image frame center coordinate for each oblique image included in the filtered subset (Balfour: paragraph 47, The other 80% of the final heuristic score is preferably based on the delta distance that the center point of each image (contained in the geo-polys (22) data) in the immediate neighbor list (21) is from the current surface reference point (18)); see rationale for rejection of claim 5.
With respect to claim 10, Lawler et al. as modified by Aizawa and Balfour disclose the method of claim 9, wherein determining the calculated straight line distance is skipped for each oblique image of the plurality of non-composite oblique aerial images that is not included in the filtered subset (Balfour: paragraph 47, a preferred embodiment ranks all images in the immediate neighbor list (21) of the currently displayed oblique image by the images' heading deviance from the current viewpoint (17), preferably counted as 20% of the final heuristic score, and preferably eliminating any neighbor image with a heading deviance greater than 30 degrees); see rationale for rejection of claim 5. The calculation is skipped for the eliminated neighbor images that have been filtered.
With respect to claim 15, Lawler et al. as modified by Aizawa and Balfour disclose the system of claim 13 for executing the method of claims 4-5; see rationale for rejection of claims 4-5.
Response to Arguments
Applicant's arguments filed June 25, 2026 have been fully considered but they are not persuasive. Applicant argues that Lawler does not teach “determining user-input-based view perspective information or selecting among oblique aerial images based on that information” in that Lawler’s metadata corresponds to an already-obtained image (page 7 of remarks). However, Lawler discloses the user input initiates obtaining the metadata (paragraph 36, Beginning at block 302, a request is received from a computer user, such as computer user 101, for an image. At block 304, the requested image is identified and obtained from a data store. At block 306, metadata (as described above) is obtained). Furthermore, the projection matrix from the metadata comprises view perspective information (paragraph 33, The projection matrix provides the basis for locating a geographic coordinate within an image taken at an oblique angle. Indeed, the projection matrix comprises translation information and/or formulae that enable a location to be plotted on an image from an oblique angle, give the latitude, longitude, and elevation of that point).
Lawler is not cited to teach “selecting among oblique aerial images based on that information,” or “compare parameters of that view perspective information to parameters associated with each oblique aerial image to select a best match oblique image.” In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). This limitation is taught by Aizawa (Office Action, pages 3-4).
Applicant argues that the Lawler metadata describes how an already-requested or already-obtained image relates to geographic coordinates so that external geographic data can be displayed using that image (bottom of page 7). However, the claimed “view perspective information” does not preclude Lawler from determining said view perspective information from metadata in response to user input for an image.
Applicant argues that Aizawa does not disclose “comparing, selecting, and outputting of a best match oblique image” in that Aizawa selects a physical image capture apparatus, not a best match oblique image (page 8). However, by selecting the image capture apparatus and displaying the corresponding image to the image capture apparatus, this selects the oblique image that is displayed.
Applicant argues that Aizawa does not teach or suggest the “claimed comparison against parameters associated with each non-composite oblique aerial image of a geographic area” (bottom of page 8). However, the parameters of the image capture apparatuses used to generate the oblique images are directly correlated to the oblique images due to intrinsic camera parameters. Thus, by comparing the parameters of the image capture apparatuses, Aizawa compares parameters of the images generated by said image capture apparatuses.
Applicant argues that the rationale for combining Lawler and Aizawa is too general (page 9). In response to applicant's argument that “The Office Action does not explain why a person of ordinary skill in the art would have modified Lawler's image-request and metadata-overlay system to incorporate Aizawa's camera-path, virtual-viewpoint, and physical camera-selection process,” the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). Aizawa provides the benefit of selecting a viewpoint closer to the desired viewpoint, which would be applicable to viewing images in the Lawler system.
Applicant argues that the Office Action does not explain how the proposed modification would result in the claimed method as a whole (bottom of page 9). However, as previously explained, Aizawa does not need to be bodily incorporated into Lawler. Furthermore, the teaching of obtaining images from the viewpoint selected by Aizawa is applicable when combined with the Lawler system.
Applicant argues that Balfour does not provide an articulated reason to modify Lawler and Aizawa to arrive at claim 1 (page 10). However, Balfour is not cited to teach an articulated reason to modify Lawler and Aizawa.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW GUS YANG whose telephone number is (571)272-5514. The examiner can normally be reached M-F 9 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW G YANG/Primary Examiner, Art Unit 2614
7/31/26
Prosecution Insights