Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
1. This action is responsive to communications: Application filed on June 26, 2026, and Drawings filed on June 26, 2026.
2. Claims 1–29, 31, 33, 58, 59 are pending in this case. Claim 1 is the independent claims.
3. Applicant’s election without traverse of 1-29, 31, 33, 58, 59 in the reply filed on 6/26/2026 is acknowledged.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Allowable Subject Matter
Claim 3, 6, 11, 17, 23, 24 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-29, 31, 33, 58, 59 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 claim the limitation of outputting the object label, the anchor points, and at least one of the cuboid in the real world coordinates of the real 3D space, a centroid of the cuboid, or the bounding box of the object with at least one of the images, for generating a 3D map which includes the object located in the real world coordinates in a virtual 3D space.
It is unclear whether this is a conjunction claim limitation or a disjunction claim limitation. Applicant is using both “or” and “and” in the claim making it difficult to interpret which aspect needs to be outputted. For the purpose of a compact prosecution, it is interpreted that this is a disjunction claim limitation.
Applicant claim the limitation “receiving at least one image which includes an object.” But applicant also claims the limitation “wherein a first one of the images is captured from a first camera device and a second one of the images is captured from a second camera device.”
Since the claim only claims one or more images, it is unclear where the second one of the images come from whether it is still part of the at least one image. It is unclear howe the second camera device captures a second one of the images if there is only one image. It is unclear whether the claim requires that the system receive at least 2 images. For the purpose of a compact prosecution that it is interpreted that the system receives at least two images.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 5, 7, 14, 16, 18, 19, 21, 22, 25, 26, 29, 32, 33, 58, 59 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu, Pub. No.: 20220012504.
With regard to claim 1:
Lee discloses a localization method, comprising: receiving at least one image which includes an object (See paraph 69 wherein objects can be detected in digital images); generating for each image, using a positioning module: a camera location in real world coordinates of real 3-Dimensional (3D) space (see fig. 15 and paragraph 154 for camera coordinate system with camera location with respect to location of the objects or vehicles), a camera orientation (see paragraph 72 wherein the orientation includes a pitch angle of the camera.), and a camera distance to the object (see paragraph 248 wherein the system determines a distance from a center of a camera used to capture the image to the sample point of the vehicle or object); generating, using an image 2D object detection module and each image: i) an object label of the object detected in that image (see paragraph 73 and fig. 1A wherein the object is labelled as car), ii) a bounding box of the object in that image(see paragraph 72 and fig. 1A wherein the object is in the bounding box), and iii) feature points in that image (see paragraph 98 for points X.sub.O,1, X.sub.O,2, and X.sub.O,3); generating, using a cuboid generator, the bounding box for each image (see paragraph 72 and fig. 1A wherein the object is in the bounding box),, the camera location for each image (see fig. 15 and paragraph for camera coordinate system with camera location with respect to location of the objects or vehicles), the camera orientation for each image (see paragraph 72 wherein the orientation includes a pitch angle of the camera.), and the camera distance to the object for each image(see paragraph 248 wherein the system determines a distance from a center of a camera used to capture the image to the sample point of the vehicle or object): a cuboid (see paragraph 136 and fig. 12C for cuboid with coordinates) in the real world coordinates of the real 3D space (see paragraph 78 wherein The geometric relationship between 2D coordinates on an image (including 2D image coordinate 301) and the corresponding 3D points (including 3D point coordinate 302) in the 3D real-world space (here, the camera coordinate system) are shown) which bounds the object in the real world coordinates of the real 3D space (see paragrpah78 wherein Knowledge of the general camera geometry can be utilized for performing PnP and other techniques. In the pinhole camera model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation. ); generating, using an anchor point generator, the feature points of the at least one image, and the cuboid: anchor points in the real world coordinates of the real 3D space of the object which are contained in the cuboid (see paragraph 150 anchor points in fig. 11a and 11B, In FIG. 15B, the locations of 3D bounding boxes and 3D sample points (or keypoints) in the camera coordinate system are shown in the image 1501. ); wherein a first one of the images is captured from a first camera device and a second one of the images is captured from a second camera device (see paragraph 115 wherein in some cases, the one or more images 901 can be captured by one or more cameras included in the system 900 or included outside of the system 900 in the computing device comprising the system 900.”).
Lee does not disclose the aspect of outputting the object label, the anchor points, and at least one of the cuboid in the real world coordinates of the real 3D space, a centroid of the cuboid, or the bounding box of the object with at least one of the images, for generating a 3D map which includes the object located in the real world coordinates in a virtual 3D space.
However Liu discloses outputting the object label, the anchor points, and at least one of the cuboid in the real world coordinates of the real 3D space, a centroid of the cuboid, or the bounding box of the object with at least one of the images, for generating a 3D map which includes the object located in the real world coordinates in a virtual 3D space (See fig. 8 for 3d map See fig. 9 and paragraph 63 wherein 914 show the object with bounding box is displayed and paragraph 65 shows the step of display identification information of the object see fig. 2 and paragraph 33 for displaying principle or anchor points). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Liu to Lee so the user can also be informed about the objects in the images in order to make better decisions such as maneuvering traffic in a driving situation with visual guide.
With regard to claim 4:
Lee and Liu discloses The localization method of claim 1, wherein the generating the cuboid in the real world coordinates of the real 3D space includes transforming the cuboid from camera 3D coordinates to the real world coordinates of the real 3D space (Lee see fig 12C and paragraph 78 Knowledge of the general camera geometry can be utilized for performing PnP and other techniques. In the pinhole camera model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation. FIG. 3 is a diagram illustrating an example of the pinhole camera model. The geometric relationship between 2D coordinates on an image (including 2D image coordinate 301) and the corresponding 3D points (including 3D point coordinate 302) in the 3D real-world space (here, the camera coordinate system) are shown).
With regard to claim 5:
Lee and Liu disclose The localization method of claim 1, wherein the generating the anchor points in the real world coordinates of the real 3D space includes transforming the feature points in the respective image to the anchor points in camera 3D coordinates and transforming the anchor points in the camera 3D coordinates to the real world coordinates of the real 3D space (Lee see paragrpah78 and fig 3 and fig. 4 In the pinhole camera model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation. FIG. 3 is a diagram illustrating an example of the pinhole camera model. The geometric relationship between 2D coordinates on an image (including 2D image coordinate 301) and the corresponding 3D points (including 3D point coordinate 302) in the 3D real-world space (here, the camera coordinate system) are shown.).
With regard to claim 7:
Lee and Liu disclose The localization method of claim 1, further comprising: generating, using a pose estimation module, the at least one image, the camera location, the camera orientation, the camera distance to the object, and the bounding box of the object in each image: a pose of the object in the real world coordinates of the real world coordinates of the real 3D space (Lee See paragraph 69 For instance, object localization can be performed on a single, monocular image to estimate a six-dimensional (6D) pose parameter (including a 3D translation vector and a 3D rotation vector, providing the six dimensions) in the camera coordinate system of an object in the image.); and outputting the pose of the object for the generating the 3D map which includes the object having the pose in the real world coordinates in the virtual 3D space (Liu See fig. 8 for 3d map See fig. 9 and paragraph 63 wherein 914 show the object with its pose and with bounding box is displayed and 918 and paragraph 65 shows the step of display identification information of the object). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Liu to Lee so the user can also be informed about the objects in the images in order to make better decisions such as maneuvering traffic in a driving situation with visual guide.
With regard to claim 14:
Lee and Liu disclose The localization method of claim 7, wherein the generating the pose of the object in the real world coordinates of the real 3D space further uses the anchor points (Lee see paragraph 150 anchor points in fig. 11a and 11B, In FIG. 15B, the locations of 3D bounding boxes and 3D sample points (or keypoints) in the camera coordinate system are shown in the image 1501. ); in the real world coordinates of the real 3D space of the object which are contained in the cuboid (Lee See paragraph 69 For instance, object localization can be performed on a single, monocular image to estimate a six-dimensional (6D) pose parameter (including a 3D translation vector and a 3D rotation vector, providing the six dimensions) in the camera coordinate system of an object in the image.).
With regard to claim 16:
Lee and Liu disclose the localization method of claim 1, wherein the at least one image, the camera location, and the camera orientation is received from a third party mapping service (Lee sea paragraph 108 wherein some cases, the system 900 can include one or more cameras (not shown) that can be used to capture the one or more images 901. In some cases, the system 900 can receive, retrieve, or otherwise obtain the one or more images 901 from other sources (e.g., from another device, from storage, from a network-based location such as a server over the Internet, or other source).).
With regard to claim 18:
Lee and Liu disclose the localization method of claim 1, wherein the outputting does not output a 3D model or point cloud map of the object (wherein Lee teaches the aspect processing image but does not output object a 3D model or point cloud map of the object).
With regard to claim 19:
Lee and Liu disclose the localization method of claim 1, wherein the at least one image includes a plurality of images (Lee See paraph 69 wherein objects can be detected in digital images).
With regard to claim 21:
Lee and Liu disclose the localization method of claim 1, wherein the positioning module includes a global positioning system (GPS), a local positioning system (LPS), and/or a Light Detection And Ranging (LiDAR) scanner (See Liu paragraph 76 In one or more arrangements, the one or more vehicle sensors 1221 can include one or more accelerometers, one or more gyroscopes, an inertial measurement unit (IMU), a dead-reckoning system, a global navigation satellite system (GNSS), a global positioning system (GPS).). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Liu to Lee so the user can also be informed about the objects in the images in order to make better decisions such as maneuvering traffic in a driving situation with visual guide and to use GPS to accurately identify the locations of the object.
With regard to claim 22:
Lee and Liu disclose The localization method of claim 1, further comprising performing, using a mapping module, the object label, the anchor points, and the at least one of the cuboid, the centroid, or the bounding box of the object with at least one of the at least one image: the generating of the 3D map which includes the object located in the real world coordinates in the virtual 3D space (Lee see paragraph 86 and fig. 4 wherein The pinhole camera model 400 can be used to map three-dimensional, real-world coordinates to the two-dimensional coordinate system 407 of the image plane. For example, a point P 422 on an object 420 in the real world can have 3D coordinates [X, Y, Z]. The point P 422 can be projected or mapped to a point p 424, whose 2D coordinates within the image plane are [x, y]. ).
With regard to claim 25:
Lee and Liu disclose The localization method of claim 22, wherein the generating of the 3D map includes the mapping module retrieving, using the object label: a 3D model of the object; wherein the 3D map includes the 3D model of the object in the real world coordinates in the virtual 3D space (Lee see fig. 10 and paragraph 112 wherein The process of FIG. 10 illustrates a 1-Point RANSAC-based pose estimation technique. For example, given a 3D model 1007 of a vehicle 1011 (from the one or more 3D models 907) including a set of vertices, the system 900 can compute a 6-dimensional (e.g., having 6-degrees-of-freedom or 6-DoF) pose hypothesis of the object in an image 1001 using a sample point 1014 (selected by sample point selection engine 904) and a 2D bounding box 1012 shown in the image 1001. ).
With regard to claim 26:
Lee and Liu disclose The localization method of claim 22, wherein the mapping module is in a camera device that captured the at least one image (Lee see paragraph 73 The object localization can be used to estimate a 6D pose parameter (3D translation vector and 3D rotation vector) of the two cars in the camera coordinate system. FIG. 1C and FIG. 1D are images 102 and 103 (from a rear-facing camera of a tracking vehicle) illustrating another example of object detection and object localization results, with 3D bounding boxes shown in the image 102 and the 3D meshes shown in image 103.).
With regard to claim 29:
Lee and Liu discloses The localization method of claim 1, wherein: the positioning module includes a positioning model that includes a first convolutional neural network (CNN); and/or the image 2D object detection module includes an image 2D object detector model that includes a second CNN (Lee see paragraph 74 wherein various techniques can be used to perform 3D object localization. In some examples, direct localization methods can use neural networks (e.g., convolutional neural networks (CNNs) or other neural network based system or algorithm) to regress the road object state).
With regard to claim 32:
Lee and Liu discloses The localization method of claim 1, wherein the localization method is performed by a camera device that captured the at least one image (Lee see paragraph 78 wherein The geometric relationship between 2D coordinates on an image (including 2D image coordinate 301) and the corresponding 3D points (including 3D point coordinate 302) in the 3D real-world space (here, the camera coordinate system) are shown. A set of formulas 304 are also shown in FIG. 3, which can be used for projection of the 3D point coordinate 302 (and/or other 3D point coordinates on the 3D object 308) onto the image plane 306 of a 2D image. As shown, a 2D homogenous image coordinate x (or point) is equal to the product of a camera projection matrix P and a 3D homogenous world coordinate X (or point).).
With regard to claim 33:
Lee and Liu discloses The localization method of claim 1, wherein the localization method is performed by at least one processor (Lee See paragraph 225 wherein The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, an application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. )
Claim 58 is rejected for the same reason as claim 1.
Claim 59 is rejected for the same reason as claim 1.
Claims 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu and further in view of Ahmadyan et al. No. 2022/0191542A1.
With regard to claim 2:
Lee and Liu do not disclose the localization method of claim 1, further comprising: generating, using a centroid generator and the cuboid: the centroid of the cuboid in real world coordinates of the real 3D space.
However Ahmadyan discloses The localization method of claim 1, further comprising: generating, using a centroid generator and the cuboid: the centroid of the cuboid in real world coordinates of the real 3D space (See fig. 7 and paragraph 76 the bounding volume may be a cuboid (which may alternatively be referred to as a rectangular prism or a box), and vertex coordinates 622-624 may thus represent the coordinates of eight corner vertices corresponding to corners of the cuboid and/or at least one center vertex corresponding to a center of the cuboid. Specifically, bounding volume 2D coordinates 620 may represent the position of a projection of each of the vertices of the 3D cuboid onto 2D space of image 604.) It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Ahmadyan to Lee and Liu so the system can determine the centroid of the cuboid in order to understand the center of the object and provide the user with useful information and to better understand the attributes of the object.
Claims 8 and 9 and 10 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu and further in view of Chumerin Pub. No.: 20240005525A1.
With regard to claim 8:
Lee and Liu do not disclose The localization method of claim 7, further comprising: generating, using a front detection module, the object label, the bounding box for each image, and the at least one image: front identifying information of the object; and wherein the generating the pose of the object in the real world coordinates of the real 3D space further uses the front identifying information of the object.
However Chumerin discloses the aspect of generating, using a front detection module, the object label, the bounding box for each image, and the at least one image: front identifying information of the object (see abstract the system obtaining the position in the image of a rectangular bounding box surrounding the vehicle, obtaining the position of a vertical split line of the bounding box configured to be placed at a corner of the vehicle separating a first portion of the vehicle from a second portion of the vehicle, the first and second portion being either the front, the back, or a side of the vehicle, ; and wherein the generating the pose of the object in the real world coordinates of the real 3D space further uses the front identifying information of the object (See abstract the system determines a signed horizontal shift of a horizontal position, associated with the vertical split line, with respect to a horizontal reference position, associated to the bounding box, normalizing the signed horizontal shift to obtain a value θ to be used in the representation of the pose of the vehicle.). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Chumerin to Lee and Liu so the system can identify the front of the object and use that information to accurately determine the pose of the object and keep the user informed about the objects in the images.
With regard to claim 9:
Lee and Liu and Chumerin The localization method of claim 8, wherein the front identifying information includes: a point of view of a 3D model of the object, a front bounding box of a front of the object, an image of the front of the object, a 3D model or point cloud map of only the front of the object, the anchor points of the front of the object, or descriptive text of the front of the object. (see abstract the system obtaining the position in the image of a rectangular bounding box surrounding the vehicle, obtaining the position of a vertical split line of the bounding box configured to be placed at a corner of the vehicle separating a first portion of the vehicle from a second portion of the vehicle, the first and second portion being either the front, the back, or a side of the vehicle, determining a signed horizontal shift of a horizontal position, associated with the vertical split line, with respect to a horizontal reference position, associated to the bounding box, normalizing the signed horizontal shift to obtain a value θ to be used in the representation of the pose of the vehicle.). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Chumerin to Lee and Liu so the system can identify the front of the object and use that information to accurately determine the pose of the object and keep the user informed about the objects in the images.
With regard to claim 10
Lee and Liu and Chumerin disclose The localization method of claim 7, further comprising: retrieving, using the object label and an object database: front identifying information of the object; and wherein the generating the pose of the object in the real world coordinates of the real 3D space further uses the front identifying information of the object (see abstract the system obtaining the position in the image of a rectangular bounding box surrounding the vehicle, obtaining the position of a vertical split line of the bounding box configured to be placed at a corner of the vehicle separating a first portion of the vehicle from a second portion of the vehicle, the first and second portion being either the front, the back, or a side of the vehicle, determining a signed horizontal shift of a horizontal position, associated with the vertical split line, with respect to a horizontal reference position, associated to the bounding box, normalizing the signed horizontal shift to obtain a value θ to be used in the representation of the pose of the vehicle.). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Chumerin to Lee and Liu so the system can identify the front of the object and use that information to accurately determine the pose of the object and keep the user informed about the objects in the images.
With regard to claim 15
Lee and Liu and Chumerin disclose The localization method of claim 1, further comprising: generating, using a front detection module, front identifying information which identifies a face of the cuboid as being a front of the object; and wherein the generating the 3D map uses the front identifying information of the object (Chumerin see abstract the system obtaining the position in the image of a rectangular bounding box surrounding the vehicle, obtaining the position of a vertical split line of the bounding box configured to be placed at a corner of the vehicle separating a first portion of the vehicle from a second portion of the vehicle, the first and second portion being either the front, the back, or a side of the vehicle, determining a signed horizontal shift of a horizontal position, associated with the vertical split line, with respect to a horizontal reference position, associated to the bounding box, normalizing the signed horizontal shift to obtain a value θ to be used in the representation of the pose of the vehicle.). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Chumerin to Lee and Liu so the system can identify the front of the object and use that information to accurately determine the pose of the object and keep the user informed about the objects in the images.
Claim 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu and further in view of Ahmed, Pub. No.: 11694345B2.
With regard to claim 12:
Lee and Liu do not disclose the localization method of claim 7, wherein the generating of the 3D map includes determining, using a mapping module, a change in the pose and updating the object already in the 3D map with the changed in the pose
However Ahmed discloses the aspect wherein the generating of the 3D map includes determining, using a mapping module, a change in the pose (see paragraph 8 column 2 line 10 to line 37 The system deriving (i) an object motion and (ii) a scene motion by applying a motion tracking algorithm separately to the object features and the scene features in the image data sequence; tracking, along the image data sequence with respect to the camera, (i) an object pose of the object and (ii) a scene pose of the scene, by applying a 3D tracking algorithm respectively to the object features and the scene features) and updating the object already in the 3D map with the changed in the pose (See paragraph 8 column 2 line 10 to line 37 outputting the object pose as a second pose of the object in one image of the image data sequence when a difference between the object motion and the scene motion is greater than a threshold in the one image; and outputting the scene pose as the second pose of the object in the one image when the difference is equal to or less than the threshold in the one image.). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Ahmad to Lee and Liu so the user would receive updated information about the pose change of the object to be more informed and respond accordingly.
Claims 13 and 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu and further in view of Abbott, 20200035122 A1.
With regard to claim 13:
Lee and Liu do not disclose the localization method of claim 7, further comprising determining that the pose is different than a stored pose of the object and outputting an instruction to move the object in the real 3D space to the stored pose
However Abbott discloses the aspect of determining that the pose is different than a stored pose of the object (see paragraph 33 the current position and orientation can be used to calculate at least two of six degrees of freedom (6DoF) error based on a difference between the current position and orientation of the physical object and a target pose of the physical object. ) and outputting an instruction to move the object in the real 3D space to the stored pose (See paragraph 33 A virtual representation of the object and/or a coordinate frame representing the current position and orientation of the physical object can be output to a display to provide a visualization of a pose of the physical object in 3D space. Also, a translation cue and/or a rotation cue can be output to the display to indicate a direction to move the physical object to align the physical object with the target pose. ). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Abbott to Lee and Liu so the user is provided guidance on the stored pose of the object, how it differ from the current pose of the object and provide the user with guidance on how the pose can be realigned.
With regard to claim 27:
Lee and Liu and Abbott disclsoe The localization method of claim 1, further comprising determining that the cuboid (Lee see figure 1and fig. 12C for cuboid ) or the centroid is different than a location of a stored cuboid or stored centroid of the object (Abbott see paragraph 33 the current position and orientation can be used to calculate at least two of six degrees of freedom (6DoF) error based on a difference between the current position and orientation of the physical object and a target pose of the physical object. ) and outputting an instruction to move the object in the real 3D space to the location of the stored cuboid or the stored centroid. (Abbott See paragraph 33 A virtual representation of the object and/or a coordinate frame representing the current position and orientation of the physical object can be output to a display to provide a visualization of a pose of the physical object in 3D space. Also, a translation cue and/or a rotation cue can be output to the display to indicate a direction to move the physical object to align the physical object with the target pose. ). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Abbott to Lee and Liu so the user is provided guidance on the stored position of the object, how it differ from the current position of the object and provide the user with guidance on how the pose can be realigned.
Claim 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu and further in view of CHAMBERLIN, 20190317974 A1.
With regard to claim 20:
Lee and Liu do not disclose the localization method of claim 1, wherein the object label is unique to the object.
However Chamberlin discloses The localization method of claim 1, wherein the object label is unique to the object (See paragraph 228 wherein Street view 1202 depicts a building, where tags 1204A-B are mapped between virtual coordinates of the virtual 3D grid to real world coordinates, for example, defined by longitude 1206, latitude 1208, and altitude 1210. 3D virtual grid may be defined according to cubes 1202 each having a unique ID, for example cube 1202 has a unique ID of 1348.) It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Chamberlin to Lee and Liu so the user can recognize the objects based on their individual unique id to be more informed about the objects.
Claim 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu and further in view of Nemirovsky, 20220057224 A1
With regard to claim 28:
Lee and Liu do not disclose the localization method of claim 1, further comprising displaying the 3D map on a display device.
However Nemirovsky discloses the aspect of displaying the 3D map on a display device (See Abstract for generating three-dimensional map data to be displayed as a virtual trail overlay on a camera display of the mobile device, processing by a visualization application on the mobile device, such that an augmented reality virtual trail overlay is displayed on the camera display of the mobile device. ). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Nemirovsky to Lee and Liu so the user can better understand the context of the image based displayed 3D map that provide visual information that help better understand the contents in the image.
Claim 31 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Pub. No.: 2021/0209797A1, in view of Liu and further in view Herman et al., 20200331465A1.
With regard to claim 31:
Lee and Liu do not disclose the localization method of claim 1, wherein the first camera device and the second camera device are each a stationary camera device.
However Herman discloses the aspect wherein the first camera device and the second camera device are each a stationary camera device. (See paragraph 33Traffic infrastructure systems can include stationary cameras continuously recording roadways 202. Host vehicle could potentially include V-to-X data exchange with static objects like buildings that broadcast or transmit a location or locations, via a network such as the “Internet of things,” for example, to help populate HD maps 300.”). It would have been obvious to one of ordinary skill in the art, at the time the filing was made to apply Herman to Lee and Liu so the system use stationary camera to capture images for high quality and stable images that are unlikely to have high artifact or blurriness to provide the user and system with clear information about the objects.
Pertinent Arts
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yu, Pub No.: 20230082420 A1 discloses a method for the editing of web pages by selectively providing editing logic and data that associates portions of the page with data sources used to provide the portions.
Marzorati, Pub. No.: US 20240096046 A1, discloses a method of displaying digital media content (e.g., electronic books) on physical surfaces or objects. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. For example, a process can include receiving, by an extended reality device, a request to display media content on a display surface.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DI XIAO whose telephone number is (571)270-1758. The examiner can normally be reached 9Am-5Pm est M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at (571) 272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DI XIAO/Primary Examiner, Art Unit 2178
Prosecution Insights