DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 11, 2026 has been entered.
Claims 1-26 are pending in this case. Claims 1, 5, 11, and 21 have been newly amended. Claim 26 has been newly added. No claims have been newly cancelled. This action is made Non-Final.
Claim Objections
Claims 5 and 26 are objected to because of the following informalities:
Claim 5 recites, “…wherein receiving the mesh geometry representation of the object includes…” but should recite, “…wherein retrieving the mesh geometry representation of the object includes…”
Claim 26 recites, “...retrieving…in the geographic indexing system…find a data structure in the geographic indexing system…” but should recite, “...retrieving…in the geographic database…find a data structure in the geographic database…”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5-7, 11, 15-17, 21, 23, and 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sardari et al. (US 2020/0312042).
As to claim 1, Sardari et al. disclose a method (e.g. process 300 of Figure 3, implemented three-dimensional reconstruction system 100 of Figure 1A or interactive computing system 160 further comprising three-dimensional reconstruction system 168 of Figure 1B), comprising: receiving, from a camera (e.g. camera 102) associated with an application (e.g. associated with an application of interactive computing system 160, e.g. game application(s) 156 and/or application host systems 164), an image of an environment (e.g. an image of a surrounding area)(step 302, [0121] notes the three-dimensional reconstruction system, e.g. via camera 102, can retrieve, receive, or otherwise obtain an image and geographic coordinates from a device, e.g. geolocation of the user’s phone from a global positioning system component on the mobile phone, where [0052], [0053] notes camera 102 can capture an image and/or video of a surrounding area, e.g. a user may capture a particular view of an area via his/her mobile phone); determining, based on the image of the environment (e.g. determining, based on the image of the surrounding area), a location of an object in the environment (e.g. a location of an object in the surrounding area)(step 308 (may be performed prior to step 304, e.g. after step 302), [0124] notes the three-dimensional reconstruction system can input the image data into a machine learning module and receive an output of the machine learning module that can include information and/or characteristics of the image, e.g. indications of an object and/or object type in the image, and indications of location and/or relative location of the objects, (see also at least [0054]-[0056], [0058], [0069], [0070], and [0091])); retrieving a mesh geometry representation of the object from a region in a geographic database (e.g. retrieving a mesh (and/or texture) of the object in a database, e.g. database of feature data 112, 182 and/or database of created assets 118, 184) and identified based on the location of the object (e.g. identified based on the location of the object)(step 306, [0122] notes the three-dimensional system can determine whether a database has data, such as feature data (e.g. structure, lake data), corresponding to the identified location of the user, where the feature data may be searched for via a mapping service or other service or database that associates features and their real world dimensions to specific GPS locations, and additionally or alternatively the three-dimensional reconstruction system may check a database for information previously stored as a result of implementing a similar process by a prior user, and if the three-dimensional reconstruction system can retrieve the feature data, at step 310, [0125], the three-dimensional reconstruction system can access (and/or generate) three-dimensional data, e.g. a three-dimensional mesh, e.g. of a building, and apply a texture onto the three-dimensional mesh (see also at least [0059]-[0062], [0093], [0094], [0097], [0098], [0101]-[0104])); and providing the mesh geometry representation of the object to the application (e.g. providing the mesh (and/or applied texture) of the object, e.g. three-dimensional representation of the object, as a rendered two-dimensional view of a three-dimensional scene to one of game application(s) 156 and/or application host systems 164)(steps 312, [0126] notes three-dimensional reconstruction system may modify the three-dimensional representations of the objects and/or features, step 314, [0127] notes the three-dimensional reconstruction system can arrange the generated three-dimensional representation of objects and/or features into a virtual three-dimensional scene, step 316, [0128] notes the three-dimensional reconstruction system can render a two-dimensional view of the three-dimensional scene, and step 320, [0129], [0130] notes the three-dimensional reconstruction system can store the three-dimensional scene with the three-dimensional objects and/or features in a database for later use, e.g. by a future player, where [0082]-[0086] further notes the three-dimensional reconstruction system can obtain data associated with a game application from the application host systems 164 and can provide object and/or feature data to the application host systems 164 and/or for storage in the data store 166, the application host systems 164 and game engine 178 further to execute each of game applications 156A, 156B (see also at least [0067], [0068], [0096])).
As noted above, Sardari et al. describes its three-dimensional reconstruction system for performing the method as outlined in Figure 3, which may be implemented within a system in communication with multiple applications, e.g. game applications 156 and application host systems 164. Further noted above, the mesh geometry representation may be further rendered as part of a three-dimensional scene that may be provided to one or more of the applications described. Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to recognize that the mesh geometry representation as the rendered scene may be provided to the application as described, yielding predictable results, without changing the scope of the invention.
As to claim 5, Sardari et al. disclose receiving the mesh geometry representation of the object includes: sending the location of the object to a façade service; and receiving, from the façade service (e.g. from a mapping service), a data structure representing the region, the region representing a section of the Earth (e.g. feature data, further including a mesh and/or texture representing an object in a region according to real-world dimensions to specific GPS coordinates), the data structure including the mesh geometry representation of the object (e.g. the feature data including the mesh of the object) (e.g. as noted in claim 1, step 306, [0122] notes the three-dimensional system can determine whether a database has data, such as feature data (e.g. structure, lake data), corresponding to the identified location of the user, where the feature data may be searched for via a mapping service or other service or database that associates features and their real world dimensions to specific GPS locations (e.g. region of the Earth), and additionally or alternatively the three-dimensional reconstruction system may check a database for information previously stored as a result of implementing a similar process by a prior user, and if the three-dimensional reconstruction system can retrieve the feature data, at step 310, [0125], the three-dimensional reconstruction system can access (and/or generate) three-dimensional data, e.g. a three-dimensional mesh, e.g. of a building, and apply a texture onto the three-dimensional mesh).
As to claim 6, Sardari et al. disclose the object includes a building and terrain (e.g. Figure 3B, [0131] illustrates building, e.g. Empire State Building of New York, e.g. Figure 3C, [0132] further illustrates city view including Empire State Building of New York with terrain).
As to claim 7, Sardari et al. disclose the mesh geometry representation of the building is separate from the mesh geometry representation of the terrain (e.g. Figure 3B, [0131] illustrates building, e.g. Empire State Building of New York, separate from, e.g. Figure 3C, [0132] which further illustrates city view including Empire State Building of New York with terrain).
As to claim 11, Sardari et al. disclose a computer program product comprising a non-transitory storage medium (Figure 7, game media 12, storage 40, read-only memory (ROM) 46, random access memory (RAM) 48), the computer program product including code (e.g. program code) that, when executed by processing circuitry (e.g. processing unit 20)(e.g. [0148], [0149] notes processing unit 20, where [0154]-[0156] notes program code stored in game media 12, storage 40, ROM 46, RAM 48), causes the processing circuitry to perform a method (e.g. computing device 10, implementing at least the three-dimensional reconstruction systems of Figures 1A and 1B, to perform the process of Figure 3), the method comprising the method as outlined in claim 1. Please see the rejection and rationale of claim 1.
Claims 15-17 are similar in scope to claims 5-7, respectively, and are therefore rejected under similar rationale.
As to claim 21, Sardari et al. disclose a system (Figure 7, computing device 10, implementing at least the three-dimensional reconstruction systems of Figures 1A and 1B), comprising: memory (e.g. game media 12, storage 40, read-only memory (ROM) 46, random access memory (RAM) 48); and processing circuitry coupled to the memory (e.g. processing unit 20)(e.g. [0148], [0149] notes processing unit 20, where [0154]-[0156] notes program code stored in game media 12, storage 40, ROM 46, RAM 48), the processing circuitry being configured to perform the method (e.g. process of Figure 3) as outlined in claim 1. Please see the rejection and rationale of claim 1.
Claim 23 is similar in scope to claim 5, and is therefore rejected under similar rationale.
As to claim 26, Sardari et al. disclose retrieving the mesh geometry representation of the object from the region in the geographic indexing system identified based on the location of the object includes: performing a lookup operation to find a data structure in the geographic indexing system (e.g. searching the database, e.g. via a mapping service, to retrieve feature data, including a mesh and/or texture of the object), the data structure including bounds containing the location of the object and the mesh geometry representation of the object (e.g. the feature data, further including a mesh and/or texture representing an object in a region according to real-world dimensions to specific GPS coordinates)(e.g. as noted in claim 1, step 306, [0122] notes the three-dimensional system can determine whether a database has data, such as feature data (e.g. structure, lake data), corresponding to the identified location of the user, where the feature data may be searched for via a mapping service or other service or database that associates features and their real world dimensions to specific GPS locations, and additionally or alternatively the three-dimensional reconstruction system may check a database for information previously stored as a result of implementing a similar process by a prior user, and if the three-dimensional reconstruction system can retrieve the feature data, at step 310, [0125], the three-dimensional reconstruction system can access (and/or generate) three-dimensional data, e.g. a three-dimensional mesh, e.g. of a building, and apply a texture onto the three-dimensional mesh).
Claim(s) 2-4, 12-14, and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sardari et al. (US 2020/0312042) as applied to claims 1, 11, and 21 above, and further in view of Pearson et al. (US 12,014,433).
As to claim 2, Sardari et al. the mesh geometry representation of the object, but do not disclose, but Pearson et al. disclose the mesh geometry representation of the object has a level of detail (LOD) indicative of a number of edges of the mesh geometry representation of the object (e.g. Figures 6A and 6B illustrate exterior of virtualized model of the object, e.g. building, Figure 7 illustrates interior of virtualized model of the building, Figure 8 illustrates a particular room of virtualized model of the building, Figures 9, 10, 12A, and 12B illustrate roof view of the virtualized model of the building, where Figures illustrate control table, e.g. touchscreen table, for controlling display of the different views of virtualized model of the building, where column 18, lines 5-11 and lines 60-67 notes house toggles 640/740 presented on touch screen table 615/715, which allows user to select through the options of seeing the whole house, individual floors, individual rooms, and/or the roof, thus considered “a level of detail indicative of a number of edges”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Sardari et al.’s mesh geometry representation of the object, e.g. a building, with Pearson et al.’s mesh geometry representation of the object having levels of details such that the object may be displayed with different property characteristics to allow for easier user navigation and manipulation (see column 17, lines 30 thru column 18, lines 33).
As to claim 3, Sardari et al. modified with Pearson et al. disclose the LOD is specified by the application (modified with Pearson, e.g. as noted in claim 2, Figures illustrate control table, e.g. touchscreen table, for controlling display of the different views of virtualized model of the building, which, as noted in claim 1, may be performed via user device 130 with application 132).
As to claim 4, Sardari et al. modified with Pearson et al. disclose the LOD is represented by a first number and a second number, where the first number indicates the number of edges of the object and the second number indicates a number of semantic features of the object (modified with Pearson, e.g. as noted in claim 2, Figures illustrate a number of views at different levels of the object, e.g. the building as a whole, the interior of the building, the roof of the building, and further illustrate a number of sematic features, e.g. labels for different parts of the exterior and/or interior of the building).
Claims 12-14 are similar in scope to claims 2-4, respectively, and are therefore rejected under similar rationale.
Claim 22 is similar in scope to claim 2, and is therefore rejected under similar rationale.
Claim(s) 8, 18, and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sardari et al. (US 2020/0312042) as applied to claims 1, 11, and 21 above, and further in view of Toh et al. (US 2020/0349350).
As to claim 8, Sardari et al. disclose determining the location of the object (e.g. as noted in claim 1), but do not disclose, but Toh et al. disclose determining the location of the object includes: sending the image of the environment to a visual positioning system (VPS); and receiving the location of the object from the VPS ([0056] notes images captured with the camera assembly 212 may also be used by the AR localization engine 224 to determine a location and orientation of the mobile device 110 within a physical space, such as an interior space (e.g., an interior space of a building), based on a representation of that physical space that is received from the memory 260 or an external computing device, where the representation of a physical space may include visual features of the physical space (e.g., features extracted from images of the physical space), location-determination data associated with those features that can be used by a visual positioning system to determine location and/or position within the physical space based on one or more images of the physical space, and/or a three-dimensional model of at least some structures within the physical space).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Sardari et al.’s method of determining a location of the object using global positioning system (GPS) with Toh et al.’s method of using visual positioning system (VPS) as GPS may not be available and/or sufficiently accurate in some situations, thus providing a more reliable and accurate system (see [0019] of Toh et al.).
Claims 18 and 24 are similar in scope to claim 8, and are therefore rejected under similar rationale.
Claim(s) 9, 19, and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sardari et al. (US 2020/0312042) as applied to claims 1, 11, and 21 above, and further in view of KUBISCH et al. (US 2014/0168242).
As to claim 9, Sardari et al. disclose providing the mesh geometry representation of the object to the application (e.g. as noted in claim 1), but do not disclose, but KUBISCH et al. disclose providing a pointer to a buffer in which the mesh geometry representation of the object is stored to the application (Figure 3, [0036] notes to render a scene, draw commands are implemented, where each draw command includes a set of one or more draw calls, a given draw command typically associated with a particular graphics object within a graphics scene that is being rendered, [0037] notes before a scene is rendered, software application 125 has to set up the graphics scene, e.g. via driver program 130, specify the different shader input buffers 308 needed to store different types of shader input data associated with the graphics scene and define the command buffer 304 for draw commands that are to be executed to render the scene, where the software application 125 also passes the shader input data associated with the graphics scene to the driver program 130 and causes the driver program 130 to store the shader input data associated with the graphics scene in the appropriate shader input buffers 308, where the driver program 130 further computes pointers to the different shader input buffers 308 and passes the pointers to back to the software application 125).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Sardari et al.’s method of providing the mesh geometry representation of the object to the application to include providing a pointer to a buffer to the application as described in KUBISCH et al. such that the application may locate the object stored with readiness and ease for subsequent use, e.g. rendering (see [0037] and [0038] of KUBISCH et al.).
Claims 19 and 25 are similar in scope to claim 9, and are therefore rejected under similar rationale.
Claim(s) 10 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sardari et al. (US 2020/0312042) as applied to claims 1 and 11 above, and further in view of Kennedy et al. (US 11,016,303).
As to claim 10, Sardari et al. disclose receiving of images of environments from the camera associated with the application (e.g. as noted in claim 1), but do not disclose, but Kennedy et al. disclose providing a toggle that, when activated, stops a receiving of images of environments from the camera associated with the application (Figure 3, column 38-60 notes muting event module 312 receives camera muting events to mute or unmute one or more camera of headset 100, depending on the current state of the cameras, which provides an indication to the camera mute system 300 to toggle, e.g. activate or deactivate, image capture of the one or more cameras of the headset 100, e.g. a user manually pressing a physical or virtual button on the headset or when the headset enters a particular geographic location).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Sardari et al.’s method of receiving image of environments from a camera with Kennedy et al.’s method of “camera muting” to toggle, e.g. deactivate, image capture of one or more cameras to provide a user with the option to address privacy issues with “always on” cameras, e.g. when cameras are used in public settings, thus enhancing the functionality of the system (see Background and Summary of Kennedy et al.).
Claim 20 is similar in scope to claim 10, and is therefore rejected under similar rationale.
Response to Arguments
Applicant's arguments filed June 11, 2026 have been fully considered but they are not persuasive. Applicant amends independent claims 1, 11, and 21 to similarly recite, “…retrieving a mesh geometry representation of the object from a region in a geographic database and identified based on the location of the object…” Applicant argues on pages 9 and 10 of the Amendment filed that the prior art of record fails to teach or suggest the limitations of the claims as now amended.
In reply, in light of the amendments of the claims, newly found reference, Sardari et al. (US 2020/0312042) is used to teach the limitations of independent claims 1, 11, and 21 as now amended. Please see the rejection and rationale of the claims above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Liu et al. (US 10,332,309) disclose a system and method of identifying buildings in textured 3D mesh data and generating 3D building models.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACINTA M CRAWFORD whose telephone number is (571)270-1539. The examiner can normally be reached 8:30a.m. to 4:30p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACINTA M CRAWFORD/Primary Examiner, Art Unit 2617
Prosecution Insights