DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation - 35 USC § 101
The limitation “generating, by the computing system, a map portion based on the plurality of subgraphs” is considered a practical application of creating a map of the various image data of a vehicle traveling along a path.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 5-8, 11-13, 15-18, 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kitaura et al. (US 2020/0134866)(Hereinafter referred to as Kitaura).
Regarding claim 1, Kitaura teaches A computer-implemented method (A position estimation system includes one or more memories and one or more processors configured to acquire a first imaging position measured at a time of imaging a first image among a plurality of images imaged in time series, perform, based on a feature of the first image, calculation of a second imaging position of the first image, and perform, in accordance with a constraint condition that reduces a deviation between the first imaging position and the second imaging position, correction of at least one of the second imaging position or a three-dimensional position of a point included in the first image calculated based on the feature of the first image. See abstract), comprising:
receiving, by a computing system, image data obtained by a plurality of vehicles traveling along different trajectories within a geographical area (In V-SLAM, imaging position and orientation estimation of all the image frames is performed, and a main image frame is called a key frame ("KF"). In many cases, only the KF's imaging position and orientation is estimated with detailed analysis technique using environmental map while performing update adjustment such as addition of a feature point group and position change of the environmental map itself so that there is no contradiction between the imaging position and orientation both globally and locally. For remaining other image frames, which is not the KF, the imaging position and orientation are easily estimated using the relative relationship from the KF without updating the environmental map. See paragraph [0035]);
constructing, by the computing system, a graph representing relationships among subsets of the image data, wherein nodes of the graph correspond to subsets of the image data and edges between nodes represent similarity between the subsets (KF groups sharing feature points with the new KF are added to the pose graph as nodes that may change the position and orientation, and as the number of shared feature points increases, an edge that maintains a strong relative position and orientation is set between KFs that share feature point groups (step S1303). See paragraph [0218])( Next, among the KFs not in the pose graph, the KFs whose time difference with the new KF is within the threshold are added to the pose graph as a node that does not change the position and orientation, and among the other registered KFs, as the number of shared feature points and the number of shared feature points having the largest number of shared feature points is larger, an edge is set that keeps the relative position and orientation stronger (step S1304). See paragraph [0219]);
partitioning, by the computing system, the graph into a plurality of subgraphs based on the similarity between the subsets (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. The loop detection and dosing unit 541 performs either local BA using the KF group in the vicinity of the new KF or global BA using the entire KF group to the new KF at the time of loop occurrence to adjust the positional relationship ofKF when traveling at the same place. The. KF group in the vicinity of the new KF may be selected from the shared state of the map feature points or the like, or the shared state with the KF group when traveling at the same place in the past may be used. See paragraphs [0230]-[0231]);
and generating, by the computing system, a map portion based on the plurality of subgraphs (The 3D map feature point updating unit 531 in charge of the environmental map creation (local mapping) processing function 530 performs the removal determination of the recently added 3D map point using the added KF, as in the V-SLAM of the related art, and performs new 3D map point addition processing. See paragraph [0204])( The KF pose and feature point map optimization unit 533 in charge of the environmental map creation (local mapping) processing function 530 performs general graph optimization calculation using the two new pose graphs newly generated by the graph restriction generating unit 532. See paragraph [0223]).
Regarding claim 2, Kitaura teaches the method of claim 1, wherein the similarity between the subsets is based on at least one of location, pose, or visual properties of the image data in the subsets (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230]).
Regarding claim 3, Kitaura teaches the method of claim 1, further comprising segmenting the image data into the subsets by: determining geographical location of image frames in the image data using metadata including GPS coordinates or inertial measurement unit (IMU) data; grouping the image frames into subsets based on corresponding geographical locations falling within a predefined spatial radius; and filtering the subsets based on directional pose similarity or visual similarity (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230])( As a related art, there is a technique of postcorrecting the position of the position and orientation of the moving object calculated by the SLAM technique with reference to information acquired by global navigation satellite system (GNSS) or the like when estimating the position of the moving object based on an image imaged by a camera mounted on the moving object using the SLAM technique. See paragraph [0004])( Pose graph generation in the graph restriction generating unit 532 of the present system 500 and the graph optimization processing using the pose graph performed prior to the local BA in the KF orientation and feature point map optimization unit 533 may be performed for all new KFs, but as described above, may be performed only when the new KF has a GNSS position. For example, when some image frames of the input video has the GNSS position, the image with the GNSS position may be positively determined as the KF, as described above, in the KF updating unit 522, only when the section without the GNSS position ends and the GNSS position is newly obtained (steps S1204 and S1205 in FIG. 12), and the correction processing of the actual coordinate environmental map (position and orientation of KF group and 3D position of feature point group) 550 may be performed (step S1301 in FIG. 13A) using the obtained GNSS position without fail. See paragraph [0227]).
Regarding claim 5, Kitaura teaches The method of claim 1, wherein the generating the map portion based on the plurality of subgraphs comprises: determining one or more overlapping poses between two or more images represented in each subgraph; selecting at least one set of images represented in each subgraph based at least in part on the one or more overlapping poses; determining a deviation of at least a portion of the at least one selected set of images; and constructing the map portion using the at least one set of images (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230])(corresponding features).
Regarding claim 6, Kitaura teaches The method of claim 1, wherein the image data is captured by one or more limited field of view image sensors on the plurality of vehicles traveling, and the one or more limited field of view image sensors comprise at least one of: a single viewpoint camera, a camera with a fixed field of view, or a camera with less than a 360-degree field of view in one plane (Above all, V-SLAM is a technique that is able to use a video imaged by an in-vehicle camera as an input to estimate and create an own vehicle traveling route ( own vehicle position and orientation) and a surrounding environmental map (3D position map of image feature point group of surrounding subjects, hereinafter referred to as environmental map) using changes in the subject appearing in the imaged video. The vehicle position and orientation from the video of the ordinary vehicle may be estimated. See paragraph [0029])(planar camera, see figure 11A, less than 360 degrees.).
Regarding claim 7, Kitaura teaches The method of claim 1, further comprising: aligning a plurality of generated map portions, comprising the generated map portion, to construct a global map (In V-SLAM, imaging position and orientation estimation of all the image frames is performed, and a main
image frame is called a key frame ("KF"). In many cases, only the KF's imaging position and orientation is estimated with detailed analysis technique using environmental map while performing update adjustment such as addition of a feature point group and position change of the environmental map itself so that there is no contradiction between the imaging position and orientation both globally and locally. See paragraph [0035]).
Regarding claim 8, Kitaura teaches The method of claim 7, wherein the aligning the plurality of the generated map portions comprises: determining alignment between the plurality of the generated map portions by identifying overlapping or neighboring map portions; performing a constraints-based optimization process to refine the alignment between map portions; and integrating aligned map portions into the global map (In the present implementation, in order to reflect both the change in appearance due to image analysis and the change in GNSS position without difficulty in the optimi zation calculation of the pos1t10n and orientation of the V-SLAM key frame (KF) and the feature point group position of the surrounding environmental map, two-stage processing is performed: first, roughly adjust the GNSS position by position and orientation optimization of KFs (FIG. 3) using GNSS position, and sequentially perform optimization (FIG. 4) of both the position and orientation of KFs and the feature point group of the environmental map again using the GNSS position to match the GNSS position in detail. See paragraph [0047]).
Regarding claim 11, Kitaura teaches A non-transitory computer readable storage medium storing instructions that, when executed by a computing device, cause the computing device to perform operations (The program stored in the memory 602 causes the CPU 601 to execute coded processing by being loaded into the CPU 601. See paragraph [0100]) (A position estimation system includes one or more memories and one or more processors configured to acquire a first imaging position measured at a time of imaging a first image among a plurality of images imaged in time series, perform, based on a feature of the first image, calculation of a second imaging position of the first image, and perform, in accordance with a constraint condition that reduces a deviation between the first imaging position and the second imaging position, correction of at least one of the second imaging position or a three-dimensional position of a point included in the first image calculated based on the feature of the first image. See abstract) comprising:
receiving image data obtained by a plurality of vehicles traveling along different trajectories within a geographical area (In V-SLAM, imaging position and orientation estimation of all the image frames is performed, and a main image frame is called a key frame ("KF"). In many cases, only the KF's imaging position and orientation is estimated with detailed analysis technique using environmental map while performing update adjustment such as addition of a feature point group and position change of the environmental map itself so that there is no contradiction between the imaging position and orientation both globally and locally. For remaining other image frames, which is not the KF, the imaging position and orientation are easily estimated using the relative relationship from the KF without updating the environmental map. See paragraph [0035]);
constructing a graph representing relationships among subsets of the image data, wherein nodes of the graph correspond to subsets of the image data and edges between nodes represent similarity between the subsets (KF groups sharing feature points with the new KF are added to the pose graph as nodes that may change the position and orientation, and as the number of shared feature points increases, an edge that maintains a strong relative position and orientation is set between KFs that share feature point groups (step S1303). See paragraph [0218])( Next, among the KFs not in the pose graph, the KFs whose time difference with the new KF is within the threshold are added to the pose graph as a node that does not change the position and orientation, and among the other registered KFs, as the number of shared feature points and the number of shared feature points having the largest number of shared feature points is larger, an edge is set that keeps the relative position and orientation stronger (step S1304). See paragraph [0219]);
partitioning the graph into a plurality of subgraphs based on the similarity between the subsets (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. The loop detection and dosing unit 541 performs either local BA using the KF group in the vicinity of the new KF or global BA using the entire KF group to the new KF at the time of loop occurrence to adjust the positional relationship ofKF when traveling at the same place. The. KF group in the vicinity of the new KF may be selected from the shared state of the map feature points or the like, or the shared state with the KF group when traveling at the same place in the past may be used. See paragraphs [0230]-[0231]); and
generating a map portion based on the plurality of subgraphs (The 3D map feature point updating unit 531 in charge of the environmental map creation (local mapping) processing function 530 performs the removal determination of the recently added 3D map point using the added KF, as in the V-SLAM of the related art, and performs new 3D map point addition processing. See paragraph [0204])( The KF pose and feature point map optimization unit 533 in charge of the environmental map creation (local mapping) processing function 530 performs general graph optimization calculation using the two new pose graphs newly generated by the graph restriction generating unit 532. See paragraph [0223]).
Regarding claim 12, Kitaura teaches The non-transitory computer readable storage medium of The non-transitory computer readable storage medium of wherein the similarity between the subsets is based on at least one of location, pose, or visual properties of the image data in the subsets (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230]).
Regarding claim 13, Kitaura teaches The non-transitory computer readable storage medium of claim 11, wherein the operations further comprise segmenting the image data into the subsets by: determining geographical location of image frames in the image data using metadata including GPS coordinates or inertial measurement unit (IMU) data; grouping the image frames into subsets based on corresponding geographical locations falling within a predefined spatial radius; and filtering the grouped subsets based on directional pose similarity or visual similarity (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230])( As a related art, there is a technique of postcorrecting the position of the position and orientation of the moving object calculated by the SLAM technique with reference to information acquired by global navigation satellite system (GNSS) or the like when estimating the position of the moving object based on an image imaged by a camera mounted on the moving object using the SLAM technique. See paragraph [0004])( Pose graph generation in the graph restriction generating unit 532 of the present system 500 and the graph optimization processing using the pose graph performed prior to the local BA in the KF orientation and feature point map optimization unit 533 may be performed for all new KFs, but as described above, may be performed only when the new KF has a GNSS position. For example, when some image frames of the input video has the GNSS position, the image with the GNSS position may be positively determined as the KF, as described above, in the KF updating unit 522, only when the section without the GNSS position ends and the GNSS position is newly obtained (steps S1204 and S1205 in FIG. 12), and the correction processing of the actual coordinate environmental map (position and orientation of KF group and 3D position of feature point group) 550 may be performed (step S1301 in FIG. 13A) using the obtained GNSS position without fail. See paragraph [0227]).
Regarding claim 15, Kitaura teaches The non-transitory computer readable storage medium of claim 11, wherein the generating the map portion based on the plurality of subgraphs comprises: determining one or more overlapping poses between two or more images represented in each subgraph; selecting at least one set of images represented in each subgraph based at least in part on the one or more overlapping poses; determining a deviation of at least a portion of the at least one selected set of images; and constructing the map portion using the at least one set of images (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230])(corresponding features).
Regarding claim 16, Kitaura teaches A computing system for generating a three-dimensional map comprising a processor and a memory storing instructions that, when executed by the processor, cause the computing system to perform operations (The program stored in the memory 602 causes the CPU 601 to execute coded processing by being loaded into the CPU 601. See paragraph [0100]) (A position estimation system includes one or more memories and one or more processors configured to acquire a first imaging position measured at a time of imaging a first image among a plurality of images imaged in time series, perform, based on a feature of the first image, calculation of a second imaging position of the first image, and perform, in accordance with a constraint condition that reduces a deviation between the first imaging position and the second imaging position, correction of at least one of the second imaging position or a three-dimensional position of a point included in the first image calculated based on the feature of the first image. See abstract)comprising:
receiving image data obtained by a plurality of vehicles traveling along different trajectories within a geographical area(In V-SLAM, imaging position and orientation estimation of all the image frames is performed, and a main image frame is called a key frame ("KF"). In many cases, only the KF's imaging position and orientation is estimated with detailed analysis technique using environmental map while performing update adjustment such as addition of a feature point group and position change of the environmental map itself so that there is no contradiction between the imaging position and orientation both globally and locally. For remaining other image frames, which is not the KF, the imaging position and orientation are easily estimated using the relative relationship from the KF without updating the environmental map. See paragraph [0035]);
constructing a graph representing relationships among subsets of the image data, wherein nodes of the graph correspond to subsets of the image data and edges between nodes represent similarity between the subsets(KF groups sharing feature points with the new KF are added to the pose graph as nodes that may change the position and orientation, and as the number of shared feature points increases, an edge that maintains a strong relative position and orientation is set between KFs that share feature point groups (step S1303). See paragraph [0218])( Next, among the KFs not in the pose graph, the KFs whose time difference with the new KF is within the threshold are added to the pose graph as a node that does not change the position and orientation, and among the other registered KFs, as the number of shared feature points and the number of shared feature points having the largest number of shared feature points is larger, an edge is set that keeps the relative position and orientation stronger (step S1304). See paragraph [0219]);
partitioning the graph into a plurality of subgraphs based on the similarity between the subsets(Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. The loop detection and dosing unit 541 performs either local BA using the KF group in the vicinity of the new KF or global BA using the entire KF group to the new KF at the time of loop occurrence to adjust the positional relationship ofKF when traveling at the same place. The. KF group in the vicinity of the new KF may be selected from the shared state of the map feature points or the like, or the shared state with the KF group when traveling at the same place in the past may be used. See paragraphs [0230]-[0231]); and
generating a map portion based on the plurality of subgraphs (The 3D map feature point updating unit 531 in charge of the environmental map creation (local mapping) processing function 530 performs the removal determination of the recently added 3D map point using the added KF, as in the V-SLAM of the related art, and performs new 3D map point addition processing. See paragraph [0204])( The KF pose and feature point map optimization unit 533 in charge of the environmental map creation (local mapping) processing function 530 performs general graph optimization calculation using the two new pose graphs newly generated by the graph restriction generating unit 532. See paragraph [0223]).
Regarding claim 17, Kitaura teaches The computing system of claim 16, wherein the similarity between the subsets is based on at least one of location, pose, or visual properties of the image data in the subsets (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230]).
Regarding claim 18, Kitaura teaches The computing system of claim 16, wherein the operations further comprise segmenting the image data into the subsets by: determining geographical location of image frames in the image data using metadata including GPS coordinates or inertial measurement unit (IMU) data; grouping the image frames into subsets based on geographical locations falling within a predefined spatial radius; and filtering the grouped subsets based on directional pose similarity or visual similarity (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230])( As a related art, there is a technique of postcorrecting the position of the position and orientation of the moving object calculated by the SLAM technique with reference to information acquired by global navigation satellite system (GNSS) or the like when estimating the position of the moving object based on an image imaged by a camera mounted on the moving object using the SLAM technique. See paragraph [0004])( Pose graph generation in the graph restriction generating unit 532 of the present system 500 and the graph optimization processing using the pose graph performed prior to the local BA in the KF orientation and feature point map optimization unit 533 may be performed for all new KFs, but as described above, may be performed only when the new KF has a GNSS position. For example, when some image frames of the input video has the GNSS position, the image with the GNSS position may be positively determined as the KF, as described above, in the KF updating unit 522, only when the section without the GNSS position ends and the GNSS position is newly obtained (steps S1204 and S1205 in FIG. 12), and the correction processing of the actual coordinate environmental map (position and orientation of KF group and 3D position of feature point group) 550 may be performed (step S1301 in FIG. 13A) using the obtained GNSS position without fail. See paragraph [0227]).
Regarding claim 20, Kitaura teaches The computing system of claim 16, wherein the generating the map portion based on the plurality of subgraphs comprises: determining one or more overlapping poses between two or more images represented in each subgraph; selecting at least one set of images represented in each subgraph based at least in part on the one or more overlapping poses; determining a deviation of at least a portion of the at least one selected set of images; and constructing the map portion using the at least one set of images (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. See paragraph [0230])(corresponding features).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4, 14, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kitaura et al. (US 2020/0134866)(Hereinafter referred to as Kitaura) in view of Rogers et al. (“Normalized Graph Cuts for Visual SLAM”, IEEE, 2009)(Hereinafter referred to as Rogers).
Regarding claim 4, Kitaura teaches The method of claim 1, but is silent to wherein the partitioning the graph into the plurality of subgraphs comprises applying a recursive normalized graph-cutting algorithm to maximize intra-subgraph similarity.
Rogers teaches the ability to perform normalized graph cuts to find partitions to separate local maps to balance capacity by removing as little information as possible (Unlike in capacity based Atlas, partitioning based on normalized graph cuts requires actually splitting up an active map two generate two sub-maps rather than simply starting a new map. Fortunately, this just involves re-expressing the features which will be carried into the new map frame in its coordinate system, and transforming the covariance matrix by this transformation.)( Normalized graph cuts will be used to find partitions to separate local maps which balance map capacity while removing as little information as possible. See page 919, section II. Methodology).
Kitaura and Rogers teach of graph based maps and Rogers teaches Normalized graph cuts allow for balanced map capacity, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Kitaura with the normalized graph cuts of Rogers such that the map capacity could be balance.
Regarding claim 14, Kitaura teaches The non-transitory computer readable storage medium of claim 11, but is silent to wherein the partitioning the graph into the plurality of subgraphs comprises applying a recursive normalized graph-cutting algorithm to maximize intra-subgraph similarity.
Rogers teaches the ability to perform normalized graph cuts to find partitions to separate local maps to balance capacity by removing as little information as possible (Unlike in capacity based Atlas, partitioning based on normalized graph cuts requires actually splitting up an active map two generate two sub-maps rather than simply starting a new map. Fortunately, this just involves re-expressing the features which will be carried into the new map frame in its coordinate system, and transforming the covariance matrix by this transformation.)( Normalized graph cuts will be used to find partitions to separate local maps which balance map capacity while removing as little information as possible. See page 919, section II. Methodology).
Kitaura and Rogers teach of graph based maps and Rogers teaches Normalized graph cuts allow for balanced map capacity, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Kitaura with the normalized graph cuts of Rogers such that the map capacity could be balance.
Regarding claim 19, Kitaura teaches The computing system of claim 16, but is silent to wherein the partitioning the graph into the plurality of subgraphs comprises applying a recursive normalized graph-cutting algorithm to maximize intra-subgraph similarity.
Rogers teaches the ability to perform normalized graph cuts to find partitions to separate local maps to balance capacity by removing as little information as possible (Unlike in capacity based Atlas, partitioning based on normalized graph cuts requires actually splitting up an active map two generate two sub-maps rather than simply starting a new map. Fortunately, this just involves re-expressing the features which will be carried into the new map frame in its coordinate system, and transforming the covariance matrix by this transformation.)( Normalized graph cuts will be used to find partitions to separate local maps which balance map capacity while removing as little information as possible. See page 919, section II. Methodology).
Kitaura and Rogers teach of graph based maps and Rogers teaches Normalized graph cuts allow for balanced map capacity, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Kitaura with the normalized graph cuts of Rogers such that the map capacity could be balance.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kitaura et al. (US 2020/0134866)(Hereinafter referred to as Kitaura) in view of Williams et al. (“Real-Time SLAM Relocalisation”, 2007.)(Hereinafter referred to as Williams)
Regarding claim 10, Kitaura teaches The method of claim 1, further comprising filtering out image subsets associated with occlusions, motion blur, or other quality degradations before constructing the graph.
Williams teaches a system that can recover from tracking error (The system is easily able to detect when tracking fails and then stop the SLAM system to preserve map integrity. Tracking and mapping are only resumed when the system relocalises again using previously mapped features. See section 6.2, first paragraph)( However, current imple mentations lack the robustness required to be useful outside laboratory conditions: blur, sudden motion and occlusion all cause tracking to fail and corrupt the map. Here we present a system which automatically detects and recovers from tracking failure while preserving map integrity. See abstract).
Kitaura and Williams both teach SLAM and Williams teaches the ability to preserve the map and recover from tracking failure by automatically detecting the failure, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Kiaura with the recovery techniques of Williams such that the system could preserve the map.
Allowable Subject Matter
Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The closest prior art of record is Kitaura et al. (US 2020/0134866)(Hereinafter referred to as Kitaura).
Kitaura teaches A computer-implemented method (A position estimation system includes one or more memories and one or more processors configured to acquire a first imaging position measured at a time of imaging a first image among a plurality of images imaged in time series, perform, based on a feature of the first image, calculation of a second imaging position of the first image, and perform, in accordance with a constraint condition that reduces a deviation between the first imaging position and the second imaging position, correction of at least one of the second imaging position or a three-dimensional position of a point included in the first image calculated based on the feature of the first image. See abstract), comprising:
receiving, by a computing system, image data obtained by a plurality of vehicles traveling along different trajectories within a geographical area (In V-SLAM, imaging position and orientation estimation of all the image frames is performed, and a main image frame is called a key frame ("KF"). In many cases, only the KF's imaging position and orientation is estimated with detailed analysis technique using environmental map while performing update adjustment such as addition of a feature point group and position change of the environmental map itself so that there is no contradiction between the imaging position and orientation both globally and locally. For remaining other image frames, which is not the KF, the imaging position and orientation are easily estimated using the relative relationship from the KF without updating the environmental map. See paragraph [0035]);
constructing, by the computing system, a graph representing relationships among subsets of the image data, wherein nodes of the graph correspond to subsets of the image data and edges between nodes represent similarity between the subsets (KF groups sharing feature points with the new KF are added to the pose graph as nodes that may change the position and orientation, and as the number of shared feature points increases, an edge that maintains a strong relative position and orientation is set between KFs that share feature point groups (step S1303). See paragraph [0218])( Next, among the KFs not in the pose graph, the KFs whose time difference with the new KF is within the threshold are added to the pose graph as a node that does not change the position and orientation, and among the other registered KFs, as the number of shared feature points and the number of shared feature points having the largest number of shared feature points is larger, an edge is set that keeps the relative position and orientation stronger (step S1304). See paragraph [0219]);
partitioning, by the computing system, the graph into a plurality of subgraphs based on the similarity between the subsets (Similar to the V-SLAM of the related art, the loop detection and closing unit 541 in charge of the loop closing processing function 540 compares the image feature amounts of the entire image between the new KF and the KF image group to be held, checks the similarity, and confirms whether the user travels the same place a plurality of times (whether loop occurs) along the traveling route at the time of video acquisition. When it is determined to travel the same place with high similarity, the KF group when traveling at the same place in the corresponding past is set in the "loop KF ID" of the KF group information 551 of the related actual coordinate environmental map 550 to enable mutual reference. The loop detection and dosing unit 541 performs either local BA using the KF group in the vicinity of the new KF or global BA using the entire KF group to the new KF at the time of loop occurrence to adjust the positional relationship ofKF when traveling at the same place. The. KF group in the vicinity of the new KF may be selected from the shared state of the map feature points or the like, or the shared state with the KF group when traveling at the same place in the past may be used. See paragraphs [0230]-[0231]);
and generating, by the computing system, a map portion based on the plurality of subgraphs (The 3D map feature point updating unit 531 in charge of the environmental map creation (local mapping) processing function 530 performs the removal determination of the recently added 3D map point using the added KF, as in the V-SLAM of the related art, and performs new 3D map point addition processing. See paragraph [0204])( The KF pose and feature point map optimization unit 533 in charge of the environmental map creation (local mapping) processing function 530 performs general graph optimization calculation using the two new pose graphs newly generated by the graph restriction generating unit 532. See paragraph [0223]), but is silent to wherein partitioning the graph comprises segmenting the subsets into straight subsets and turn subsets based on deviation in orientation among images within each subset.
The prior art of record alone or in combination is silent to the limitations “wherein partitioning the graph comprises segmenting the subsets into straight subsets and turn subsets based on deviation in orientation among images within each subset” of claim 9 when read in light of the rest of the limitations in claim 9 and the claims to which claim 9 depends and thus claim 9 contains allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS R WILSON whose telephone number is (571)272-0936. The examiner can normally be reached M-F 7:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (572)-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS R WILSON/Primary Examiner, Art Unit 2611
Prosecution Insights