DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention
was made.
Claim(s) 1, 3, 9-10, 13, 15, 17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) .
Regarding claim 1, Nurminen teaches A computer-implemented method of determining a visibility of structures in an environment, the computer-implemented method comprising: receiving, by a computing system comprising one or more processors, map data comprising a plurality of three-dimensional models of structures in a physical environment ( abstract and section 5, 5.1: teaches receiving and storing a 3d city model comprising buildings and other structures used by the mobile client for rendering and navigation); determining, by the computing system, one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of a plurality of map projection cells that correspond to a plurality of locations in the physical environment (section 4.1 and 6.2: teaches computing a potentially visible set for each cell by determining which portions of the 3d scene are visible from each cell of the environment); generating, by the computing system, visibility data comprising a plurality of visibility cells associated with the plurality of map projection cells and the one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of the plurality of map projection cells ( section 6.2 and 6.3: the visibility calculations are encoded into visibility lists/pvs clusters that associate visible geometry with corresponding spatial cells, thereby generating visibility data for the environment); receiving, by the computing system, view data comprising information associated with a visual representation of the physical environment from a location of the plurality of locations in the physical environment ( section 6.4: teaches the mobile client receives the user’s current position/viewpoint and renders the corresponding view of the physical environment); determining, by the computing system, based on the view data and the visibility data, one or more visibility cells of the plurality of visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment ( Section 6.4: the runtime selects the appropriate pvs cluster and current mesh visibility list according to the user’s current location and viewpoint ); determining, by the computing system, based on point of interest data, one or more points of interest associated with the one or more visibility cells ( section 5-5.2 and section 6.2: teaches models landmarks, statues, buildings, and other salient objects, associates them with mesh groups and database information, and determines which of those objects are included within the current visibility lists/PVS for the user’s location.), but fails to teach generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location.
Murashkin teaches generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location ( Para. 31, 37, 43 and 46: teaches constructing text labels and text overlay objects associated with identified physical objects and selectively generates and displays those overlays in the current view. It would have been obvious to modify the visibility based landmark system of Nurminen to generate location aware text labels/overlay annotations as taught by Murashkin in order to provide contextual information for visible landmarks and improve user navigation and visualization of the physical environment).
Regarding claim 3, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the view data comprises a three-dimensional representation of the physical environment, and wherein surfaces of the three-dimensional representation of the physical environment are based on images of corresponding surfaces of the physical environment, and further comprising: generating, by the computing system, a virtual reality environment based on the view data comprising the three-dimensional representation of the physical environment and the one or more annotations (Nurminen, figure 1; section 5.1, penultimate paragraph, first sentence; section 5.2, second paragraph, first two sentences; section 6.4, rendering step 7; figure 6; section 8, first sentence: discloses surfaces of the three-dimensional representation of the physical environment are based on images of corresponding surfaces of the physical environment, and generating a virtual reality environment comprising the three-dimensional representation of the physical environment and the one or more annotations).
Regarding claim 9, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the view data is received from a mobile computing device comprising a camera, a smartphone, augmented reality glasses, or an extended reality headset (Nurminen: section 7.1, first paragraph - the smartphone Nokia 6630 is also a mobile computing device equipped with a camera).
Regarding claim 10, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location comprises: determining, by the computing system, an appearance of the one or more annotations based on a distance of the one or more points of interest from the map projection cell (Nurminen, section 4.4 "Levels of detail and texture management", paragraph 1), wherein the appearance of the one or more annotations comprises a size of the one or more annotations or a color of the one or more annotations (Nurminen , section 4.4 "Levels of detail and texture management", paragraph 1 - the size of the billboard representing a statue will depend on the distance from the map projection cell).
Regarding claim 13, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location comprises: determining, by the computing system, one or more locations of the one or more annotations based on one or more locations of the one or more points of interest (Nurminen, section 5.2, paragraph 4: discloses determining one or more locations of the one or more annotations based on one or more locations of the one or more points of interest. the location of a statue as POI determines the location of its billboard as annotation).
Regarding claim 15, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the structures comprise one or more buildings, one or more statues, one or more fountains, one or more gates, one or more roads, one or more trees, one or more vehicles, one or more natural formations, or one or more bridges (Nurminen, fig.2: discloses the structures comprise one or more buildings).
Regarding claim 17, Nurminen teaches One or more tangible non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising: receiving map data comprising a plurality of three-dimensional models of structures in a physical environment ( abstract and section 5, 5.1: teaches receiving and storing a 3d city model comprising buildings and other structures used by the mobile client for rendering and navigation); determining, by the computing system, one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of a plurality of map projection cells that correspond to a plurality of locations in the physical environment (section 4.1 and 6.2: teaches computing a potentially visible set for each cell by determining which portions of the 3d scene are visible from each cell of the environment); generating, by the computing system, visibility data comprising a plurality of visibility cells associated with the plurality of map projection cells and the one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of the plurality of map projection cells ( section 6.2 and 6.3: the visibility calculations are encoded into visibility lists/pvs clusters that associate visible geometry with corresponding spatial cells, thereby generating visibility data for the environment); receiving, by the computing system, view data comprising information associated with a visual representation of the physical environment from a location of the plurality of locations in the physical environment ( section 6.4: teaches the mobile client receives the user’s current position/viewpoint and renders the corresponding view of the physical environment); determining, by the computing system, based on the view data and the visibility data, one or more visibility cells of the plurality of visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment ( Section 6.4: the runtime selects the appropriate pvs cluster and current mesh visibility list according to the user’s current location and viewpoint ); determining, by the computing system, based on point of interest data, one or more points of interest associated with the one or more visibility cells ( section 5-5.2 and section 6.2: teaches models landmarks, statues, buildings, and other salient objects, associates them with mesh groups and database information, and determines which of those objects are included within the current visibility lists/PVS for the user’s location.), but fails to teach generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location.
Murashkin Teaches generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location ( Para. 31, 37, 43 and 46: teaches constructing text labels and text overlay objects associated with identified physical objects and selectively generates and displays those overlays in the current view. It would have been obvious to modify the visibility based landmark system of Nurminen to generate location aware text labels/overlay annotations as taught by Murashkin in order to provide contextual information for visible landmarks and improve user navigation and visualization of the physical environment).
Regarding claim 19, Nurminen teaches A computing system comprising: one or more processors; one or more non-transitory computer-readable media storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising: receiving map data comprising a plurality of three-dimensional models of structures in a physical environment ( abstract and section 5, 5.1: teaches receiving and storing a 3d city model comprising buildings and other structures used by the mobile client for rendering and navigation); determining, by the computing system, one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of a plurality of map projection cells that correspond to a plurality of locations in the physical environment (section 4.1 and 6.2: teaches computing a potentially visible set for each cell by determining which portions of the 3d scene are visible from each cell of the environment); generating, by the computing system, visibility data comprising a plurality of visibility cells associated with the plurality of map projection cells and the one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of the plurality of map projection cells ( section 6.2 and 6.3: the visibility calculations are encoded into visibility lists/pvs clusters that associate visible geometry with corresponding spatial cells, thereby generating visibility data for the environment); receiving, by the computing system, view data comprising information associated with a visual representation of the physical environment from a location of the plurality of locations in the physical environment ( section 6.4: teaches the mobile client receives the user’s current position/viewpoint and renders the corresponding view of the physical environment); determining, by the computing system, based on the view data and the visibility data, one or more visibility cells of the plurality of visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment ( Section 6.4: the runtime selects the appropriate pvs cluster and current mesh visibility list according to the user’s current location and viewpoint ); determining, by the computing system, based on point of interest data, one or more points of interest associated with the one or more visibility cells ( section 5-5.2 and section 6.2: teaches models landmarks, statues, buildings, and other salient objects, associates them with mesh groups and database information, and determines which of those objects are included within the current visibility lists/PVS for the user’s location.), but fails to teach generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location.
Murashkin Teaches generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location ( Para. 31, 37, 43 and 46: teaches constructing text labels and text overlay objects associated with identified physical objects and selectively generates and displays those overlays in the current view. It would have been obvious to modify the visibility based landmark system of Nurminen to generate location aware text labels/overlay annotations as taught by Murashkin in order to provide contextual information for visible landmarks and improve user navigation and visualization of the physical environment).
Claim(s) 2, 4, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of Fedosov (US-20190180512-A1) .
Regarding claim 2, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, but fails to teach wherein the view data comprises a video stream associated with the visual representation of the physical environment from a field of view of an image capture device at the location, and further comprising: generating, by the computing system, an augmented reality environment based on the view data comprising the video stream and the one or more annotations.
Fedosov teaches wherein the view data comprises a video stream associated with the visual representation of the physical environment from a field of view of an image capture device at the location ( Para. 15, 21, 68-71, 73 and 88: describes capturing an image of the real environment with a camera on a mobile device and displaying that real-world view. While it often refers to image/frames rather than explicitly saying “video stream”, it is directed to a live camera view on a mobile device), and further comprising: generating, by the computing system, an augmented reality environment based on the view data comprising the video stream and the one or more annotations ( Para. 2-5: describe ar systems overlaying computer generated information onto a view of the real environment. Para 73: captures the camera image, para 74: determines relevant POIs and para 75: overlays computer generated virtual objects onto the displayed camera view. Para 75, 90-99: discloses displaying augmented Poi’s/ annotations as computer generated visual objects over the camera view. It would have been obvious to combine the augmented reality visualization techniques of Fedosov into the system of Nurminen in view of Murashkin in order to present the determined points of interest and annotations on a live camera view of the physical environment, thereby improving user interaction and providing an intuitive real time visualization of location based information).
Regarding claim 4, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the view data comprises a two-dimensional image of the physical environment (Nurminen, Sections 5.1-5.2 and figs 1-2: Teaches the use of a 2D map and digital photographs/textures of the real environment. It also describes street-level photographs used as textures for buildings) , and further comprising: generating, by the computing system, an annotated two-dimensional image of the physical environment (Fedosov, Para 15,22, 40-45, 73,75, 93-107 and 121-123: teaches capturing an image of the real environment and overlaying computer generated indicators/virtual objects.) based on the view data comprising the two-dimensional image of the physical environment and the one or more annotations ( Fedosov , Para. 15, 40-45, 73-75, 93-107 and 121-123: the annotated display is generated from the captured camera image together with generated annotations).
Regarding claim 18, Nurminen in view of Murashkin and in further view of Fedosov teaches The one or more tangible non-transitory computer-readable media of claim 17, wherein the view data comprises a video stream associated with the visual representation of the physical environment from a field of view of an image capture device at the location (Fedosov, Para. 15, 20, 51, and 69, 88: describes displaying live camera image of the real environment on a mobile device. Para. 15, 20, 51, 69: the camera captures an image of the real environment or a part of the real environment, which is displayed while AR content is overlaid. Para.15, 20, 29, 88, 117-121: discloses a camera adapted for capturing an image of the real environment, including a mobile-device camera or head mounted display camera, and determines POIs relative to the captured camera view) .
Regarding claim 20, Nurminen in view of Murashkin in further view of Fedosov teaches The computing system of claim 19, wherein the view data comprises a video stream associated with the visual representation of the physical environment from a field of view of an image capture device at the location (Fedosov, Para. 15, 20, 51, and 69, 88: describes displaying live camera image of the real environment on a mobile device. Para. 15, 20, 51, 69: the camera captures an image of the real environment or a part of the real environment, which is displayed while AR content is overlaid. Para.15, 20, 29, 88, 117-121: discloses a camera adapted for capturing an image of the real environment, including a mobile-device camera or head mounted display camera, and determines POIs relative to the captured camera view) .
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of Golovinskiy (US-20170228926-A1) .
Regarding claim 5, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the determining, by the computing system, one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of a plurality of map projection cells that correspond to a plurality of locations in the physical environment comprises ( Nurminen, Section 4.1-4.2, 6.2, 6.4 and 7.1: teaches Potentially visible sets for cells, identifying which meshes/buildings are visible from each cell): determining, by the computing system, based on performance of one or more surface visibility operations, the one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of a plurality of map projection cells that correspond to the plurality of locations in the physical environment, but fails to teach wherein the one or more surface visibility operations comprise one or more ray casting operations or one or more ray tracing operations.
Golovinskiy Teaches wherein the one or more surface visibility operations comprise one or more ray casting operations or one or more ray tracing operations(Para. 37-38: teaches tracing a camera ray between an output image pixel and a corresponding point on the 3d model. Claim 8 and para. 40-49: teaches using camera rays to determine source images and visibility relationships. It would have been obvious to incorporate Golovinskiy ray-based visibility determination into the system of Nurminen in view of Murashkin because ray tracing/ casting provides an established technique for determining visible portions of a 3d model and can improve accuracy of visibility calculations ).
Claim(s) 6 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of Iguchi (US-20230017612-A1) .
Regarding claim 6, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the determining, by the computing system, one or more portions of the plurality of three-dimensional models of structures that are visible from each map projection cell of a plurality of map projection cells that correspond to a plurality of locations in the physical environment comprises ( Nurminen, Section 4.1 -4.2, 6.2 and 6.4: teaches partitions the environment into cells computing a potentially visible set for each cell and determines which 3d meshes/buildings are visible from that cell) but fails to teach determining, by the computing system, that the one or more portions of the plurality of three-dimensional models of structures are within a predetermined distance of each map projection cell of the plurality of map projection cells.
Iguchi Teaches determining, by the computing system, that the one or more portions of the plurality of three-dimensional models of structures are within a predetermined distance of each map projection cell of the plurality of map projection cells (para.743: teaches determining visibility for each tile from a predetermined position uses visibility information for each tile and further includes distance information between regions together with a threshold value for the visibility determination. It would have been obvious to incorporate the predetermined position and threshold based visibility determination of Iguchi into the visibility culling system of Nurminen in view of Murashkin to improve the accuracy and efficiency of selecting portions of 3d structures should be considered visible for each map projection cell).
Regarding claim 12, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location comprises (Nurminen, Section 6.4: teaches generating and rendering landmarks associated with visible objects in a 3d city model.): determining, by the computing system, based on the visibility data ( Nurminen, section 4.1, 4.2, 6.2 and 6.4: performs visibility calculations, computes visibility lists, and determines which objects, landmarks, and messages are visible before rendering . The rendering decisions are therefore based on visibility data), that the one or more annotations are within a predetermined distance of the one or more points of interest (Iguchi. Para 743: teaches visibility information together with distance information and predetermined geometric information. It describes visibility information indicating whether data is visible from a predetermined position and connectivity information including information indicating a distance of the region).
Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of Arth (made of reference in ids: ARTH ET AL: "Wide area localization on mobile phones", ISMAR 2009, 8TH IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY, 19 October 2009, XP031568941) .
Regarding claim 7, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the determining, based on the view data and the visibility data, one or more visibility cells of the plurality of visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment comprises: determining, by the computing system.
Nurminen in view of Murashkin Fails to teach wherein the view data comprises one or more two-dimensional images of the physical environment, based on inputting the view data and the visibility data into one or more machine-learned models, the one or more visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment, wherein the one or more machine-learned models are configured to determine the one or more visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment based on detection, recognition, or classification of one or more features of the one or more two-dimensional images.
Arth teaches wherein the view data comprises one or more two-dimensional images of the physical environment (fig. 1 pp 1-2 and p .7 section 3.1: explicitly uses images captured by the mobile phone camera. The offline data acquisition stage collects images and the online localization stage uses these images as input for localization), based on inputting the view data and the visibility data into one or more machine-learned models ( section 3.1-3.4 and section 4.1: uses machine learning for localization. The system extracts features from the input images and matches them against the reconstruction database using a vocabulary tree/approximate nearest neighbor search), the one or more visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment ( Arth, p.1, p.6-7 sections 3.2-3.3: teaches a learned model producing an estimated 6DOF pose which corresponds to a location in the physical environment.), wherein the one or more machine-learned models are configured to determine the one or more visibility cells that are associated with the map projection cell that corresponds to the location in the physical environment based on detection, recognition, or classification of one or more features of the one or more two-dimensional images ( Arth p.6 section 3.4, p.5 section 3.1 and p.6 section 3.2: detects and describe local image features in the 2D images and matches them to features in the 3D model database using a learned visual vocabulary tree. Pose estimation is then computed based on these detected features. It would have been obvious to a person of ordinary skill to combine the teachings of Arth learned image based localization to obtain users location into the system of Nurminen in view of Murashkin to identify the visibility cells associated with that location.).
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of TAKACS (made of reference in ids: TAKACS ET AL: "Outdoors augmented reality on mobile phone using loxel- based visual feature organization", PROCEEDINGS OF THE MIR WORKSHOP ON MULTIMEDIA INFORMATION RETRIEVAL, VANCOUVER, 30 October 2008, XP058201597) and KITAYAMA (US-20250265827-A1) .
Regarding claim 8, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, further comprising: generating, by the computing system, based on inputting a plurality of images of the physical environment into one or more machine-learned models, the point of interest data (Nurminen pages 11-14, section 5.1-5.2, 6.4 and fig.5: teaches generating point of interest information such as landmarks, buildings, parks, statues, routes and areas of interest for the 3D environment), but fails to teach wherein the one or more machine-learned models are configured to generate the point of interest data based on detection, recognition, or classification of one or more features of the plurality of images that correspond to one or more points of interest.
Takacs Teaches wherein the one or more machine-learned models are configured to generate the point of interest data based on detection (abstract, fig.1, section 1.3, fig.2, section 2.1: discloses an image recognition pipeline that extracts features matching and recognition, and returns information associated with the recognized object and nearby points of interest), recognition, or classification of one or more features of the plurality of images that correspond to one or more points of interest (Fig.1 section 1.3, 2.1, and 2.1.1-2.1.2: describes recognizing objects from captured images matching image features, assigning labels, and retrieving associated information based on the recognition result. It would have been obvious to incorporate the image recognition techniques of Takacs into the location based 3d mapping system of Nurminen in view of Murashkin to automatically recognize features in captured images and generate point of interest information, thereby improving the accuracy and automation of identifying landmarks.) Takacs fails to explicitly teaches a machine learning model.
KITAYAMA teaches a machine learning model for feature extraction and determining points of interest( para.115: teaches a machine learning model being used for feature extraction and then used for recognition/determination of a point of interest in an image. It would have been obvious to employ the machine learning model based feature extraction and image recognition techniques of Kitayama into the system of Nurminen in view of Murashkin and Takacs to improve recognition accuracy and identifying points of interest from images.).
Claim(s) 11 is rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of Amon (US-20220076117-A1) and Wang (US-20210256261-A1).
Regarding claim 11, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, wherein the generating, by the computing system, one or more annotations based on the one or more points of interest that are visible from the map projection cell associated with the location comprises: but fails to teach determining, by the computing system, one or more transient objects that occlude the one or more points of interest, wherein the one or more transient objects comprise one or more vehicles, foliage, one or more temporary signs, or one or more pedestrians; and determining, by the computing system, that the one or more annotations are in a location that does not include the one or more transient objects.
Amon teaches determining, by the computing system, one or more transient objects that occlude the one or more points of interest, wherein the one or more transient objects comprise one or more vehicles, foliage, one or more temporary signs, or one or more pedestrians (Para.3: teaches detecting objects, including persons and vehicles and addresses partially occluded objects in real world scenes. It would have been obvious to incorporate the object detection techniques of Amon into the system of Nurminen in order to detect and classify real world transient objects, such as persons and vehicles, that may occlude point of interest. Thereby improving accuracy and reliability of displaying point of interest information)) Amon fails to teach determining, by the computing system, that the one or more annotations are in a location that does not include the one or more transient objects.
Wang teaches determining, by the computing system, that the one or more annotations are in a location that does not include the one or more transient objects (Para.97: teaches repositioning annotation menu when it is occluded so it remains visible. It would have been obvious to incorporate the annotation repositioning techniques of Wang into the combined system of Nurminen in view of Murashkin and in further view of Amon so that annotations are dynamically repositioned to avoid detected occlusions ).
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of Hunt (US-20260170855-A1).
Regarding claim 14, Nurminen in view of Murashkin teaches The computer-implemented method of claim 13, but fails to teach wherein the plurality of three-dimensional models of structures are associated with one or more bounding boxes, and wherein the one or more annotations are located at one or more centroids of the one or more bounding boxes associated with the one or more points of interest.
Hunt teaches wherein the plurality of three-dimensional models of structures are associated with one or more bounding boxes ( Para.85 and fig.3: disclose 3d bounding boxes associated with detected 3d objects), and wherein the one or more annotations are located at one or more centroids of the one or more bounding boxes associated with the one or more points of interest ( fig.3 and para.85: disclose a label associated with 3d bounding box. The figure depicts the label positioned with the bounding box supporting placement relative to the center of the box. It would have been obvious to modify the annotation system of Nurminen in view of Murashkin to associate annotations with 3d bounding boxes taught by Hunt because bounding boxes provide a standardized spatial representation of three dimensional objects, allowing annotations to be positioned consistently relative to the associated objects and improving the clarity and accuracy of object identification ).
Claim(s) 16 is rejected under 35 U.S.C. 103 as being unpatentable over Nurminen et al. (made of reference in ids: ANTTI NURMINEN: "m-LOMA - a mobile 3D city map", 3D WEB TECHNOLOGY, ACM, 18 April 2006, XP058245460) in view of Murashkin (US-20250285400-A1) in further view of TSAI YI-ZHEN (made of reference in ids: TSAI YI-ZHEN ET AL: "The World is Too Big to Download: 3D Model Retrieval for World-Scale Augmented Reality", MMSys 2023 - Proceedings of the MMSys'23, 14th ACM Multimedia Systems Conference, June 7-10, 2023, Vancouver, XP059075511) .
Regarding claim 16, Nurminen in view of Murashkin teaches The computer-implemented method of claim 1, but fails to teach wherein the plurality of map projection cells comprise a plurality of S2 cells.
TSAI YI-ZHEN teaches wherein the plurality of map projection cells comprise a plurality of S2 cells( Tsai discloses partitioning geographic regions into level-17 s2 cells stating “ we define a sub-zone as 25 level-17 S2 cells”. It would have been obvious to implement the techniques of TSAI YI-ZHEN well known S2 cells into the system of Nurminen because S2 provides a well-known hierarchical geographic indexing scheme for efficiently partitioning large geographic regions).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Blumenfeld (WO 2024091226 A1): discloses technique for detecting and locating objects in a physical environment using 3D point cloud models and image data, making it relevant to processing three-dimensional representations of physical environments for spatial analysis.
Gunnar et al (US-20250173973-A1): discloses automatically detecting, identifying, and labeling objects within interactive three dimensional models of physical environments, including presenting object information during user walkthroughs.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LATRELL ANTHONY CREARY whose telephone number is (703)756-1219. The examiner can normally be reached Mon - Fri 7:30am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao WU can be reached on (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LATRELL ANTHONY CREARY/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
Prosecution Insights