DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20 of US Application No. 17/354,936 are currently pending and have been examined. Applicant amended claims 1, 19, and 20.
Response to Arguments/Amendments
The amendment filed June 6, 2025 has been entered. Claims 1-20 are currently pending in the Application. Applicant’s amendments to the claims have overcome the claim objections set forth in the Non-Final Office Action.
Applicant’s arguments with respect to claim(s) 1-20 under 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments with respect to the rejection of claims 1-20 under 35 U.S.C. 101 directed to an abstract idea have been fully considered but they are not persuasive.
The Examiner has carefully considered applicant’s arguments and respectfully disagrees. Initially, applicant argues that claim 1 recites a practical application that improves the functionality of both the computer and the technological process related to an information processing apparatus. Applicant submits that the claim is not directed towards a judicial exception and that the claim does not recite a mental process. Applicant submits that according to Example 37, the claim is patent eligible due to integrating the mental process into practical application (See pages 8-13 of Applicant’s remarks).
The Examiner has considered such arguments, but respectfully submits that the claim limitations, as currently presented, do not have to be interpreted as narrowly as argued by the applicant. Claim 1 merely recites “...acquire sensor information...”, “...generate map data...”, “...estimate the position...”, “...detect...a second measurement point...”, “...correct...a position...”, “...correct...position and orientation information...” which are claimed broadly, and can be done mentally if the person is given the collected data. For example, the recited steps of detecting when a measurement point is revisited, calculating a difference between the position of the measurement point before and after alignment, and updating position/orientation information based on that difference, are all activities that can be carried out by a human using observation and manual calculations given the data. These steps therefore fall within the “mental process” category of abstract ideas, which is a judicial exception. While applicant is arguing that the limitations improve the functionality of the computer and technological process of the information processing apparatus, the limitations “...acquire sensor information...”, “...generate map data...”, are recited at a high level of generality and amounts to mere data gathering and pre-solution actions, which are a form of insignificant extra solution activity. While applicant is arguing that the limitations improve the technological process of the apparatus, the improvement is not shown in the steps of the limitations in the claims. While applicant is arguing that “An information processing apparatus comprising: a memory storing instructions; and at least one processor executing the instructions causing the information processing apparatus to function as…”, “an acquisition unit…”, “a generation unit…”, “an estimation unit…”, “a detection unit…”, “a first correction unit…”, and “a second correction unit…” are practical applications and significantly more, such data types are 1) generically collecting data and 2) does not improve the functioning of a computer, but is merely using a generic computer (server/processor) to process the information faster. No limitation in the claim puts limits on a timeframe in which the determinations are happening or how the apparatus is controlled as a result of the determinations, and as such could be done mentally. Furthermore, Applicant’s reliance on Example 37 is not persuasive. Example 37 relates to relocating icons on a graphical user interface, which is factually distinct from the present claims directed to correcting measurement point data. Example 37 therefore does not support eligibility for the instant claims. Accordingly, the claims are directed to a mental process, which is a judicial exception, and do not include additional elements that amount to significantly more.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In January, 2019 (updated October 2019), the USPTO released new examination guidelines setting forth a two-step inquiry for determining whether a claim is directed to non-statutory subject matter. According to the guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claims 1-20 are directed toward non-statutory subject matter, as shown below:
STEP 1: Do claims 1, 19, and 20 fall within one of the statutory categories? Yes. The claims are directed toward a machine, a method including at least one step, and a machine which fall within one of the statutory categories.
STEP 2A (PRONG 1): Are the claims directed to a law of nature, a natural phenomenon or an abstract idea? Yes, the claims are directed to an abstract idea.
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
Claim 1. An information processing apparatus comprising:
a memory storing instructions;
and at least one processor executing the instructions causing the information processing apparatus to function as:
an acquisition unit configured to acquire sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a sensor configured to move;
a generation unit configured to generate map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor;
an estimation unit configured to estimate the position and orientation of the sensor based on the sensor information acquired by the acquisition unit and the measurement point;
a detection unit configured to detect, based on the output from the sensor, a second measurement point when a first measurement point is revisited;
a first correction unit configured to correct, at a position where the revisit is detected, position and orientation information of one or more third measurement points used by the estimation unit to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point;
a second correction unit configured to correct, after position and orientation information of the third measurement points have been corrected by the first correction unit, position and orientation information associated with a plurality of measurement points different from the third measurement points.
The method in claim 1, specifically the limitations emphasized above, is a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. It merely consists of estimating the position and orientation of the sensor, and detecting a second measurement point. This is equivalent to a person observing the sensor, mentally estimating the position and orientation of the sensor and mentally seeing a measurement point.
Claim 19. An information processing method comprising:
acquiring sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a sensor configured to move;
generating map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor;
estimating the position and orientation of the sensor based on the acquired sensor information and the measurement point;
detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited;
correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point;
correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points.
The method in claim 19, specifically the limitations emphasized above, is a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. It merely consists of estimating the position and orientation of the sensor, detecting a second measurement point, and correcting a position and orientation and positions and orientations. This is equivalent to a person observing the sensor, mentally estimating the position and orientation of the sensor, mentally seeing a measurement point, and mentally correcting the position and orientation and positions and orientations.
Claim 20. A non-transitory storage medium storing a program for causing a computer to execute an information processing method, the method comprising:
acquiring sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a moving sensor;
generating map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor;
estimating the position and orientation of the sensor based on the acquired sensor information and the measurement point;
detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited;
correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point;
correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points.
The method in claim 20, specifically the limitations emphasized above, is a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. It merely consists of estimating the position and orientation of the sensor, detecting a second measurement point, and correcting a position and orientation and positions and orientations. This is equivalent to a person observing the sensor, mentally estimating the position and orientation of the sensor, mentally seeing a measurement point, and mentally correcting the position and orientation and positions and orientations.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? No, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim 1. An information processing apparatus comprising:
a memory storing instructions;
and at least one processor executing the instructions causing the information processing apparatus to function as:
an acquisition unit configured to acquire sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a sensor configured to move;
a generation unit configured to generate map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor;
an estimation unit configured to estimate the position and orientation of the sensor based on the sensor information acquired by the acquisition unit and the measurement point;
a detection unit configured to detect, based on the output from the sensor, a second measurement point when a first measurement point is revisited;
a first correction unit configured to correct, at a position where the revisit is detected, position and orientation information of one or more third measurement points used by the estimation unit to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point;
a second correction unit configured to correct, after position and orientation information of the third measurement points have been corrected by the first correction unit, position and orientation information associated with a plurality of measurement points different from the third measurement points.
Claim 1 does not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. The step of “acquire sensor information…” is recited at a high level of generality and amounts to mere data gathering, which is a form of insignificant extra solution activity. Further, the step of “generate map data…” is recited at a high level of generality and amounts to mere pre-solution actions, which is a form of extra solution activity.
The limitations “An information processing apparatus comprising: a memory storing instructions; and at least one processor executing the instructions causing the information processing apparatus to function as…”, “an acquisition unit…”, “a generation unit…”, “an estimation unit…”, “a detection unit…”, “a first correction unit…”, and “a second correction unit…” are claimed generically and are operating in their ordinary capacity such that they does not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. The “An information processing apparatus comprising: a memory storing instructions; and at least one processor executing the instructions causing the information processing apparatus to function as…”, “an acquisition unit…”, “a generation unit…”, “an estimation unit…”, “a detection unit…”, “a first correction unit…”, and “a second correction unit…” merely describe how to generally “apply” the otherwise mental judgments in a generic or general purpose computing environment. The “An information processing apparatus comprising: a memory storing instructions; and at least one processor executing the instructions causing the information processing apparatus to function as…”, “an acquisition unit…”, “a generation unit…”, “an estimation unit…”, “a detection unit…”, “a first correction unit…”, and “a second correction unit…” are recited at a high level of generality and merely automate the acquiring, generating, estimating, detecting, and correcting steps. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. It should be noted that because the courts have made it clear that mere physicality or tangibility of an additional element or elements is not a relevant consideration in the eligibility analysis, the physical nature of these computer components does not affect this analysis. See MPEP 2106.05(I). Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Claim 19. An information processing method comprising:
acquiring sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a sensor configured to move;
generating map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor;
estimating the position and orientation of the sensor based on the acquired sensor information and the measurement point;
detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited;
correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point;
correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points.
Claim 19 does not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. The step of “acquiring sensor information…” is recited at a high level of generality and amounts to mere data gathering, which is a form of insignificant extra solution activity. Further, the step of “generating map data…” is recited at a high level of generality and amounts to mere pre-solution actions, which is a form of extra solution activity.
Claim 20. A non-transitory storage medium storing a program for causing a computer to execute an information processing method, the method comprising:
acquiring sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a moving sensor;
generating map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor;
estimating the position and orientation of the sensor based on the acquired sensor information and the measurement point;
detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited;
correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point;
correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points.
Claim 20 does not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. The step of “acquiring sensor information…” is recited at a high level of generality and amounts to mere data gathering, which is a form of insignificant extra solution activity. Further, the step of “generating map data…” is recited at a high level of generality and amounts to mere pre-solution actions, which is a form of extra solution activity. The limitation “A non-transitory storage medium storing a program for causing a computer to execute…” is claimed generically and is operating in its ordinary capacity such that it does not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. The “A non-transitory storage medium storing a program for causing a computer to execute…” merely describes how to generally “apply” the otherwise mental judgments in a generic or general purpose computing environment. The “A non-transitory storage medium storing a program for causing a computer to execute…” is recited at a high level of generality and merely automates the acquiring, generating, estimating, detecting, and performing steps. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer. It should be noted that because the courts have made it clear that mere physicality or tangibility of an additional element or elements is not a relevant consideration in the eligibility analysis, the physical nature of these computer components does not affect this analysis. See MPEP 2106.05(I). Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No, the claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Regarding Step 2B of the 2019 PEG, independent claims 1, 19, and 20 do not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claims do not integrate the abstract idea into a practical application.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation(s) of “An information processing apparatus comprising: a memory storing instructions; and at least one processor executing the instructions causing the information processing apparatus to function as…”, “an acquisition unit…”, “a generation unit…”, “an estimation unit…”, “a detection unit…”, “a first correction unit…”, “a second correction unit…”, and “A non-transitory storage medium storing a program for causing a computer to execute…” is/are merely means to apply the exception and do not amount to “significantly more”, as adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984, are not sufficient to amount to significantly more than the judicial exception.
Further, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field. The additional limitations of “acquire sensor information…”, “generate map data…”, “acquiring sensor information…”, “generating map data…” are well-understood, routine, and conventional activities because the specification does not provide any indication that the acquiring, generating, estimating, detecting, correcting, and performing steps are performed using anything other than a conventional computer. See also MPEP 2106.05(d)(II), and the cases cited therein, including Intellectual Ventures |, LLC v. Symantec Corp., 838 F.3d 1307, 1321 (Fed. Cir. 2016), TL! Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610 (Fed. Cir. 2016), and O/P Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015), indicate that mere performance of an action is a well-understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). Hence, the claim is not patent eligible.
CONCLUSION
Thus, since claims 1, 19, 20 are: (a) directed toward an abstract idea, (b) does not recite additional elements that integrate the judicial exception into a practical application, and (c) does not recite additional elements that amount to significantly more than the judicial exception, it is clear that claim 1 is directed towards non-statutory subject matter.
Dependent claims 2-18 further limit the abstract idea without integrating the abstract idea into practical application or adding significantly more. For example, in claim 10, the additional limitation of
“wherein the first correction unit performs correction depending on a distance between the second measurement point and the first measurement point” is an additional step that, under the broadest reasonable interpretation, covers performance of the limitation in the mind using a similar analysis applied to claims 1, 19, 20 above. The method in claim 10, specifically the limitation above, is a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. It merely consists of correcting depending on a distance between two measurement points. This is equivalent to a person looking at the environment with two points and correcting based on the view of the distance between points. Notably, the claim does not positively recite any limitations regarding the control of the unit.
As such, claims 1-20 are rejected under 35 USC 101 as being drawn to an abstract idea without significantly more, and thus are ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4-12, 14, 16, 17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kotake (US 20200012877 A1) in view of Nakazato (US 20120316820 A1).
Regarding Claim 1, Kotake teaches An information processing apparatus comprising: a memory storing instructions; and at least one processor executing the instructions causing the information processing apparatus to function as: an acquisition unit configured to acquire sensor information obtained by measuring a surrounding environment, the sensor information is obtained using an output from a sensor configured to move (See at least [0007], [0024], [0025], [0027], [0123], Fig. 2, “A program for realizing the functions of each embodiment can be stored in a storage medium such as a read-only memory (ROM) 20. The ROM 20 may also store an operating system (OS) and device drivers. A memory, such as a random access memory (RAM) 30, may temporarily store these programs.” The sensor is the camera which is configured to move and capture the scene. The obtaining unit is the acquisition unit.); a generation unit configured to generate map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor (See at least [0007], [0025], [0043], [0114], Fig. 2. “The image capturing apparatus is not fixed but can move, and can obtain a captured image of an object in the scene where the image capturing apparatus moves around…The position and orientation of the image capturing apparatus (for example, the position of the origin and the direction of the Z axis) represents the position and orientation of the capturing coordinate system with respect to a reference coordinate system (hereinafter, referred to as a world coordinate system) defined in a space (scene) where capturing is performed” and “In step S1040, the generation unit 140 generates (expands) a three-dimensional map using the position and orientation of the image capturing apparatus 170 derived in step S1030.” The sensor is the camera which is configured to move and capture the scene. The generation unit uses the camera’s position and orientation to generate the map.); an estimation unit configured to estimate the position and orientation of the sensor based on the sensor information acquired by the acquisition unit and the measurement point (See at least [0005], [0025], [0085], [0114], [0121], Fig. 2. “The image capturing apparatus is not fixed but can move, and can obtain a captured image of an object in the scene where the image capturing apparatus moves around…The position and orientation of the image capturing apparatus (for example, the position of the origin and the direction of the Z axis) represents the position and orientation of the capturing coordinate system with respect to a reference coordinate system (hereinafter, referred to as a world coordinate system) defined in a space (scene) where capturing is performed” and “For example, configuration may be taken to calculate the three-dimensional coordinates of feature points close to an assistance marker based on the position and orientation of the image capturing apparatus estimated based on the assistance marker.” The sensor is the camera which is configured to move and capture the scene. The obtaining unit is the acquisition unit. The image capturing apparatus is the estimation unit that estimates the camera’s position and orientation.).
Kotake does not explicitly disclose, however, Nakazato, in the same field of endeavor, teaches a detection unit configured to detect, based on the output from the sensor, a second measurement point when a first measurement point is revisited (See at least paragraph [0029], “To deal with the error in the depth information acquisition, the exemplary embodiment of the present invention is focused on a fact that the error in the depth information acquisition depends on the positions and the orientations of the light source and the light reception unit, and the position and the direction of the surface of the measurement target object. In other words, a correction amount of a position of a measurement point is acquired from a relationship between information about errors in the measurement (optical characteristics such as subsurface scattering and anisotropic reflection, and calibration accuracy) and an inclination of the surface of the measurement target object” and paragraph [0049], “In step S0020, a correction amount database is created. First, with respect to an arbitrary point on the surface of the measurement target object, using the depth information measured in step S0010, a difference between the positions of the point before and after the coating is acquired and registered as a correction amount in the database. Further, the position and the orientation of the measurement target object is registered in the correction amount database as a key for the database.” The system detects when a measurement point is observed again (revisited). The previously observed point corresponds to the claimed first measurement point, and the subsequently observed version corresponds to the claimed second measurement point. The system explicitly identifies the same point before and after, which aligns with detecting a second measurement point when a first measurement point is revisited.); a first correction unit configured to correct, at a position where the revisit is detected, position and orientation information of one or more third measurement points used by the estimation unit to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point (See at least paragraph [0043], “In reality, the relative positions and orientations of the light source and the camera are already defined by calibration. Accordingly, a correction amount corresponding to the position and direction of the measurement point is extracted from the correction amount database provided in advance, and the correction amount is added to the three-dimensional position of the measurement point to correct the depth information, and the corrected data is output. The method for creating the correction amount database is described with reference to the flowchart described below” and paragraph [0049], “In step S0020, a correction amount database is created. First, with respect to an arbitrary point on the surface of the measurement target object, using the depth information measured in step S0010, a difference between the positions of the point before and after the coating is acquired and registered as a correction amount in the database. Further, the position and the orientation of the measurement target object is registered in the correction amount database as a key for the database.”); and a second correction unit configured to correct, after position and orientation information of the third measurement points have been corrected by the first correction unit, position and orientation information associated with a plurality of measurement points different from the third measurement points (See at least paragraph [0044], “In the extraction of the correction amount from the database, if positions and orientations of points are close, the correction values are close too. Consequently, if the position and the direction of a key point do not exist in the correction amount database, the correction amount can be calculated by interpolating from correction amounts of points having close positions and orientations. Alternatively, the position and orientation and the correction amount of the point may be put into a polynomial or a trigonometric function as parameters to generate a correction function for returning a correction amount using the position and orientation of the point as arguments, and the value may be used” and paragraph [0093], “Similarly to the first modification in the first exemplary embodiment, the processing in steps S3020, S3030, and S3040 can be repeatedly performed to correct the depth information. In such a case, with respect to the position of the measurement point corrected in step S3040 by the depth information correction unit 240, in step S3020, the position and orientation calculation unit 220 calculates the position and the orientation of the target object again, and in step S3030, the surface information acquisition unit 230 calculates the position and the direction of the measurement point. Then, based on the newly calculated position and direction of the measurement point, in step S3040, the depth information correction unit 240 corrects the depth information of the measurement point, that is, the three-dimensional position of the measurement point. The processing is repeated to correct the depth information until the change in the position and the orientation of the target object calculated by the position and orientation calculation unit 220 in step S3020 will be a value less than or equal to a predetermined threshold.” The system distinguishes between first applying a correction amount to directly affected points (local correction) and then applying interpolation/iterative corrections across a broader set of measurement points (global correction). The sequence corresponds to the claimed requirement that the second correction unit corrects a plurality of measurement points different from the third measurement points after the first correction is complete.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Kotake with the teachings of Nakazato such that the information processing apparatus of Kotake is further configured to utilize a detection unit configured to detect, based on the output from the sensor, a second measurement point when a first measurement point is revisited, a first correction unit configured to correct, at a position where the revisit is detected, position and orientation information of one or more third measurement points used by the estimation unit to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point, and a second correction unit configured to correct, after position and orientation information of the third measurement points have been corrected by the first correction unit, position and orientation information associated with a plurality of measurement points different from the third measurement points, as taught Nakazato (See paragraph [0043], [0044], [0049], [0093].), with a reasonable expectation of success. The motivation for doing so would be increased measurement accuracy, as taught by Nakazato (See paragraph [0012].).
Regarding Claim 4, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake teaches wherein the first correction unit calculates a position and orientation at the second measurement point after correction based on a relative position and orientation between the position and orientation at the second measurement point and the position and orientation at the first measurement point (See at least paragraphs [0055], [0057], [0092], “Specifically, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame by a similar method to that in step S1030” and “In step S1230, the correction unit 160 corrects the position and orientation of the key frame using the key frame correspondence information obtained in steps S1210 and S1220” and “In one embodiment, the derivation of the position and orientation of the image capturing apparatus 170 (steps S1020 to S1030), the generation of the key frame (steps S1040 to S1050), and the correction of the three-dimensional map (step S1060) can be performed independently at any time.” The selected key frame is the first measurement point which is the position and orientation and the new key frame is the second measurement point. The steps are performed at any time.).
Regarding Claim 5, Kotake and Nakazato teach The information processing apparatus according to claim 4, as set forth in the obviousness rejection. Kotake teaches wherein the first correction unit adds a difference in position and orientation calculated based on the position and orientation at the second measurement point before correction and the position and orientation corresponding to the second measurement point after correction to a position and orientation at a third measurement point to correct the position and orientation at the third measurement point (See at least paragraphs [0055], [0057], [0092], “For example, the correction unit 160 can select all the key frames in which the angular difference in the optical axis direction (Z-axis of the capturing coordinate system) in the world coordinate system is within the threshold T.sub.Angle and the difference in position is within the threshold T.sub.Dist with respect to the new key frame. Next, the correction unit 160 derives the relative position and orientation between the key frames. Specifically, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame by a similar method to that in step S1030” and “In step S1230, the correction unit 160 corrects the position and orientation of the key frame using the key frame correspondence information obtained in steps S1210 and S1220” and “In one embodiment, the derivation of the position and orientation of the image capturing apparatus 170 (steps S1020 to S1030), the generation of the key frame (steps S1040 to S1050), and the correction of the three-dimensional map (step S1060) can be performed independently at any time.” The selected key frame is the second measurement point which is the position and orientation and the new key frame is the third measurement point. The steps are performed at any time and are performed at multiple measurement points including the third.).
Regarding Claim 6, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake teaches wherein the map data generated by the generation unit includes position information about a feature point in the surrounding environment, wherein the measurement point includes information about the feature point observed based on the sensor information at the measurement point, and wherein the estimation unit estimates the position and orientation of the sensor based on the feature point (See at least [0025], [0043], [0085], [0114], Fig. 2. “The image capturing apparatus is not fixed but can move, and can obtain a captured image of an object in the scene where the image capturing apparatus moves around…The position and orientation of the image capturing apparatus (for example, the position of the origin and the direction of the Z axis) represents the position and orientation of the capturing coordinate system with respect to a reference coordinate system (hereinafter, referred to as a world coordinate system) defined in a space (scene) where capturing is performed” and “In step S1040, the generation unit 140 generates (expands) a three-dimensional map using the position and orientation of the image capturing apparatus 170 derived in step S1030. The generation unit 140 can expand the three-dimensional map by adding a key frame to the three-dimensional map” and “For example, configuration may be taken to calculate the three-dimensional coordinates of feature points close to an assistance marker based on the position and orientation of the image capturing apparatus estimated based on the assistance marker.” The sensor is the camera which is configured to move and capture the scene which is the surrounding environment. The image capturing apparatus is the estimation unit that estimates the camera’s position and orientation.).
Regarding Claim 7, Kotake and Nakazato teach The information processing apparatus according to claim 6, as set forth in the obviousness rejection. Kotake teaches wherein the second measurement point is one or more measurement points at which the feature point used for the estimation by the estimation unit is observed, and wherein the first correction unit further corrects a position of the feature point (See at least paragraphs [0055], [0057], [0085], [0092], “Then, the relative positions and orientations of the existing key frame and the new key frame are calculated so that a color difference (for example a luminance difference) of the feature points in the existing key frame and the new key frame becomes smaller” and “For example, the correction unit 160 can select all the key frames in which the angular difference in the optical axis direction (Z-axis of the capturing coordinate system) in the world coordinate system is within the threshold T.sub.Angle and the difference in position is within the threshold T.sub.Dist with respect to the new key frame. Next, the correction unit 160 derives the relative position and orientation between the key frames. Specifically, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame by a similar method to that in step S1030” and “In step S1230, the correction unit 160 corrects the position and orientation of the key frame using the key frame correspondence information obtained in steps S1210 and S1220” and “For example, configuration may be taken to calculate the three-dimensional coordinates of feature points close to an assistance marker based on the position and orientation of the image capturing apparatus estimated based on the assistance marker” and “In one embodiment, the derivation of the position and orientation of the image capturing apparatus 170 (steps S1020 to S1030), the generation of the key frame (steps S1040 to S1050), and the correction of the three-dimensional map (step S1060) can be performed independently at any time.” The selected key frame is the second measurement point which is multiple positions and orientations. The steps are performed at any time and are performed at multiple measurement points. The image capturing apparatus is the estimation unit that estimates the camera’s position and orientation.).
Regarding Claim 8, Kotake and Nakazato teach The information processing apparatus according to claim 7, as set forth in the obviousness rejection. Kotake teaches wherein the first correction unit integrates a first feature point and a second feature point in a case where the first feature point of which a position is corrected and the second feature point included in the map data are located within a predetermined distance and a feature of the first feature point observed based on the sensor information is similar to a feature of the second feature point observed based on the sensor information. (See at least paragraphs [0052], [0055], [0057], [0085], [0092], “When it is determined that a new key frame is to be added, the generation unit 140 adds the input image as a new key frame by using the methods of Engel et al. described above. When using the method of Engel et al., feature point information for a new key frame can be generated by projecting and propagating feature point information on the immediately previous key frame (or on a neighboring key frame) onto the input image” and “Then, the relative positions and orientations of the existing key frame and the new key frame are calculated so that a color difference (for example a luminance difference) of the feature points in the existing key frame and the new key frame becomes smaller” and “In step S1220, the correction unit 160 searches for an existing key frame having a higher degree of similarity of images with respect to the new key frame. For example, the correction unit 160 selects an existing key frame such that the SSD (Sum of Squared Distance) of the luminance value between the new key frame and the existing key frame is smallest and is also less than or equal to the threshold TSSD, from among the existing key frames. Then, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame similarly to S1210” and “For example, the correction unit 160 can select all the key frames in which the angular difference in the optical axis direction (Z-axis of the capturing coordinate system) in the world coordinate system is within the threshold T.sub.Angle and the difference in position is within the threshold T.sub.Dist with respect to the new key frame. Next, the correction unit 160 derives the relative position and orientation between the key frames. Specifically, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame by a similar method to that in step S1030” and “In step S1230, the correction unit 160 corrects the position and orientation of the key frame using the key frame correspondence information obtained in steps S1210 and S1220” and “For example, configuration may be taken to calculate the three-dimensional coordinates of feature points close to an assistance marker based on the position and orientation of the image capturing apparatus estimated based on the assistance marker” and “In one embodiment, the derivation of the position and orientation of the image capturing apparatus 170 (steps S1020 to S1030), the generation of the key frame (steps S1040 to S1050), and the correction of the three-dimensional map (step S1060) can be performed independently at any time.” The selected key frame is the second measurement point which is multiple positions and orientations. The steps are performed at any time and are performed at multiple measurement points. The image capturing apparatus is the estimation unit that estimates the camera’s position and orientation. The feature points are derived from position and orientations making up key frames of the camera. Similarity between key frames is between feature points. The threshold difference is position is predetermined distance. The first and second feature points are integrated when the new key frame is added.).
Regarding Claim 9, Kotake and Nakazato The information processing apparatus according to claim 8, as set forth in the obviousness rejection. Kotake teaches wherein the first correction unit corrects the position and orientation by performing bundle adjustment on the integrated feature point and the measurement point where the integrated feature point is observed (See at least paragraphs [0052], [0057], [0085], [0092], [0122], “When it is determined that a new key frame is to be added, the generation unit 140 adds the input image as a new key frame by using the methods of Engel et al. described above. When using the method of Engel et al., feature point information for a new key frame can be generated by projecting and propagating feature point information on the immediately previous key frame (or on a neighboring key frame) onto the input image” and “In step S1230, the correction unit 160 corrects the position and orientation of the key frame using the key frame correspondence information obtained in steps S1210 and S1220” and “In one embodiment, the derivation of the position and orientation of the image capturing apparatus 170 (steps S1020 to S1030), the generation of the key frame (steps S1040 to S1050), and the correction of the three-dimensional map (step S1060) can be performed independently at any time” and “Similarly to the second embodiment, when an assistance marker is detected from an image, the correction unit 160 may derive the position and orientation of the image based on the assistance marker, and may perform bundle adjustment while fixing the position and orientation derived based on the assistance marker.” The steps are performed at any time and are performed at multiple measurement points. The feature points are derived from position and orientations making up key frames of the camera. The first and second feature points are integrated when the new key frame is added.).
Regarding Claim 10, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake teaches wherein the first correction unit performs correction depending on a distance between the second measurement point and the first measurement point (See at least paragraphs [0052], [0055], [0057], [0085], [0092], “Then, the relative positions and orientations of the existing key frame and the new key frame are calculated so that a color difference (for example a luminance difference) of the feature points in the existing key frame and the new key frame becomes smaller” and “For example, the correction unit 160 can select all the key frames in which the angular difference in the optical axis direction (Z-axis of the capturing coordinate system) in the world coordinate system is within the threshold T.sub.Angle and the difference in position is within the threshold T.sub.Dist with respect to the new key frame. Next, the correction unit 160 derives the relative position and orientation between the key frames. Specifically, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame by a similar method to that in step S1030” and “In step S1230, the correction unit 160 corrects the position and orientation of the key frame using the key frame correspondence information obtained in steps S1210 and S1220” and “In one embodiment, the derivation of the position and orientation of the image capturing apparatus 170 (steps S1020 to S1030), the generation of the key frame (steps S1040 to S1050), and the correction of the three-dimensional map (step S1060) can be performed independently at any time.” The selected key frame is the second measurement point which is multiple positions and orientations. The steps are performed at any time and are performed at multiple measurement points. The position and orientation measurement points make up key frames of the camera. The threshold difference is position is predetermined distance between frames which are the measurement points of position and orientation.).
Regarding Claim 11, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake teaches wherein the map data includes a pose graph including a relative position and orientation between measurement points, and wherein the second correction unit corrects the positions and orientations at the plurality of measurement points to minimize an error in the relative position and orientation calculated based on the relative position and orientation between measurement points illustrated in the pose graph and the positions and orientations at the plurality of measurement points (See at least paragraphs [0037], [0055], [0057], [0058], [0092], “The correction unit 160 corrects the information indicating the three-dimensional position of the feature included in the three-dimensional map based on the reliability degree of the information. For example, the correction unit 160 may update information indicating the three-dimensional position of the feature included in the three-dimensional map according to the reliability degree of the information. According to such processing, the correction unit 160 can improve consistency with the three-dimensional map. Such a process is known as optimization of a three-dimensional map (or pose graph” and “Then, the relative positions and orientations of the existing key frame and the new key frame are calculated so that a color difference (for example a luminance difference) of the feature points in the existing key frame and the new key frame becomes smaller” and “In step S1220, the correction unit 160 searches for an existing key frame having a higher degree of similarity of images with respect to the new key frame. For example, the correction unit 160 selects an existing key frame such that the SSD (Sum of Squared Distance) of the luminance value between the new key frame and the existing key frame is smallest and is also less than or equal to the threshold TSSD, from among the existing key frames. Then, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame similarly to S1210” and “For example, the correction unit 160 can select all the key frames in which the angular difference in the optical axis direction (Z-axis of the capturing coordinate system) in the world coordinate system is within the threshold T.sub.Angle and the difference in position is within the threshold T.sub.Dist with respect to the new key frame. Next, the correction unit 160 derives the relative position and orientation between the key frames. Specifically, the correction unit 160 calculates the relative position and orientation between the selected key frame and the new key frame by a similar method to that in step S1030” and “For correcting the position and orientation, it is possible to use the error between the relative position and orientation between the key frames calculated from the position and orientation in the world coordinate system that the key frames have as attributes, and the relative position and orientation between the key frames calculated in steps S1210 and S1220” and “In one embodiment, the derivation of the position and orientation of the image capturing apparatus 170 (steps S1020 to S1030), the generation of the key frame (steps S1040 to S1050), and the correction of the three-dimensional map (step S1060) can be performed independently at any time.” The steps are performed at any time and are performed at multiple measurement points. The correction unit 160 is the second correction unit.).
Regarding Claim 12, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake teaches wherein the sensor is a camera, and wherein the sensor information is an image (See at least [0007], [0025], [0027], [0123], Fig. 2. The sensor is the camera which is configured to move and capture the scene and the captured image is sensor information.).
Regarding Claim 14, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake teaches further comprising: the sensor; and a moving unit (See at least [0025], [0110], [0123], Fig. 2. The sensor is the camera which is configured to move and capture the scene and the captured image is sensor information. The sensor is fixed to the image capturing apparatus which is the moving unit.).
Regarding Claim 16, Kotake and Nakazato teach The information processing apparatus according to claim 14, as set forth in the obviousness rejection. Kotake teaches wherein the moving unit moves along a preset route based on the position and orientation estimated by the estimation unit (See at least paragraphs [0284], [0324], [0405].).
Regarding Claim 17, Kotake and Nakazato teach The information processing apparatus according to claim 14, as set forth in the obviousness rejection. Kotake teaches wherein the moving unit is a wheel or a propeller (See at least paragraphs [0159], [0179], [0343].).
Regarding Claim 19, Kotake teaches An information processing method comprising: acquiring sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a sensor configured to move (See at least [0001], [0007], [0025], [0027], [0123], Fig. 2. The sensor is the camera which is configured to move and capture the scene.); generating map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor (See at least [0007], [0025], [0043], [0114], Fig. 2. “The image capturing apparatus is not fixed but can move, and can obtain a captured image of an object in the scene where the image capturing apparatus moves around…The position and orientation of the image capturing apparatus (for example, the position of the origin and the direction of the Z axis) represents the position and orientation of the capturing coordinate system with respect to a reference coordinate system (hereinafter, referred to as a world coordinate system) defined in a space (scene) where capturing is performed” and “In step S1040, the generation unit 140 generates (expands) a three-dimensional map using the position and orientation of the image capturing apparatus 170 derived in step S1030.” The sensor is the camera which is configured to move and capture the scene. The generation unit uses the camera’s position and orientation to generate the map.); estimating the position and orientation of the sensor based on the acquired sensor information and the measurement point (See at least [0005], [0025], [0085], [0114], [0121], Fig. 2. “The image capturing apparatus is not fixed but can move, and can obtain a captured image of an object in the scene where the image capturing apparatus moves around…The position and orientation of the image capturing apparatus (for example, the position of the origin and the direction of the Z axis) represents the position and orientation of the capturing coordinate system with respect to a reference coordinate system (hereinafter, referred to as a world coordinate system) defined in a space (scene) where capturing is performed” and “For example, configuration may be taken to calculate the three-dimensional coordinates of feature points close to an assistance marker based on the position and orientation of the image capturing apparatus estimated based on the assistance marker.” The sensor is the camera which is configured to move and capture the scene. The obtaining unit is the acquisition unit. The image capturing apparatus is the estimation unit that estimates the camera’s position and orientation.).
Kotake does not explicitly disclose, however, Nakazato, in the same field of endeavor, teaches detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited (See at least paragraph [0043], “In reality, the relative positions and orientations of the light source and the camera are already defined by calibration. Accordingly, a correction amount corresponding to the position and direction of the measurement point is extracted from the correction amount database provided in advance, and the correction amount is added to the three-dimensional position of the measurement point to correct the depth information, and the corrected data is output. The method for creating the correction amount database is described with reference to the flowchart described below” and paragraph [0049], “In step S0020, a correction amount database is created. First, with respect to an arbitrary point on the surface of the measurement target object, using the depth information measured in step S0010, a difference between the positions of the point before and after the coating is acquired and registered as a correction amount in the database. Further, the position and the orientation of the measurement target object is registered in the correction amount database as a key for the database.”); correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point (See at least paragraph [0043], “In reality, the relative positions and orientations of the light source and the camera are already defined by calibration. Accordingly, a correction amount corresponding to the position and direction of the measurement point is extracted from the correction amount database provided in advance, and the correction amount is added to the three-dimensional position of the measurement point to correct the depth information, and the corrected data is output. The method for creating the correction amount database is described with reference to the flowchart described below” and paragraph [0049], “In step S0020, a correction amount database is created. First, with respect to an arbitrary point on the surface of the measurement target object, using the depth information measured in step S0010, a difference between the positions of the point before and after the coating is acquired and registered as a correction amount in the database. Further, the position and the orientation of the measurement target object is registered in the correction amount database as a key for the database.”); and correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points (See at least paragraph [0044], “In the extraction of the correction amount from the database, if positions and orientations of points are close, the correction values are close too. Consequently, if the position and the direction of a key point do not exist in the correction amount database, the correction amount can be calculated by interpolating from correction amounts of points having close positions and orientations. Alternatively, the position and orientation and the correction amount of the point may be put into a polynomial or a trigonometric function as parameters to generate a correction function for returning a correction amount using the position and orientation of the point as arguments, and the value may be used” and paragraph [0093], “Similarly to the first modification in the first exemplary embodiment, the processing in steps S3020, S3030, and S3040 can be repeatedly performed to correct the depth information. In such a case, with respect to the position of the measurement point corrected in step S3040 by the depth information correction unit 240, in step S3020, the position and orientation calculation unit 220 calculates the position and the orientation of the target object again, and in step S3030, the surface information acquisition unit 230 calculates the position and the direction of the measurement point. Then, based on the newly calculated position and direction of the measurement point, in step S3040, the depth information correction unit 240 corrects the depth information of the measurement point, that is, the three-dimensional position of the measurement point. The processing is repeated to correct the depth information until the change in the position and the orientation of the target object calculated by the position and orientation calculation unit 220 in step S3020 will be a value less than or equal to a predetermined threshold.” The system distinguishes between first applying a correction amount to directly affected points (local correction) and then applying interpolation/iterative corrections across a broader set of measurement points (global correction). The sequence corresponds to the claimed requirement that the second correction unit corrects a plurality of measurement points different from the third measurement points after the first correction is complete.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Kotake with the teachings of Nakazato such that the information processing apparatus of Kotake is further configured to utilize detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited, correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point, and correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points, as taught Nakazato (See paragraph [0043], [0044], [0049], [0093].), with a reasonable expectation of success. The motivation for doing so would be increased measurement accuracy, as taught by Nakazato (See paragraph [0012].).
Regarding Claim 20, Kotake teaches A non-transitory storage medium storing a program for causing a computer to execute an information processing method, the method comprising: acquiring sensor information obtained by measuring a surrounding environment, the sensor information is obtained using output from a moving sensor (See at least [0001], [0007], [0025], [0027], [0123], Fig. 2. The sensor is the camera which is configured to move and capture the scene.); generating map data indicating a map based on a movement path of the sensor, the map data including a measurement point where the sensor information is associated with a position and orientation of the sensor(See at least [0007], [0025], [0043], [0114], Fig. 2. “The image capturing apparatus is not fixed but can move, and can obtain a captured image of an object in the scene where the image capturing apparatus moves around…The position and orientation of the image capturing apparatus (for example, the position of the origin and the direction of the Z axis) represents the position and orientation of the capturing coordinate system with respect to a reference coordinate system (hereinafter, referred to as a world coordinate system) defined in a space (scene) where capturing is performed” and “In step S1040, the generation unit 140 generates (expands) a three-dimensional map using the position and orientation of the image capturing apparatus 170 derived in step S1030.” The sensor is the camera which is configured to move and capture the scene. The generation unit uses the camera’s position and orientation to generate the map.); estimating the position and orientation of the sensor based on the acquired sensor information and the measurement point (See at least [0005], [0025], [0085], [0114], [0121], Fig. 2. “The image capturing apparatus is not fixed but can move, and can obtain a captured image of an object in the scene where the image capturing apparatus moves around…The position and orientation of the image capturing apparatus (for example, the position of the origin and the direction of the Z axis) represents the position and orientation of the capturing coordinate system with respect to a reference coordinate system (hereinafter, referred to as a world coordinate system) defined in a space (scene) where capturing is performed” and “For example, configuration may be taken to calculate the three-dimensional coordinates of feature points close to an assistance marker based on the position and orientation of the image capturing apparatus estimated based on the assistance marker.” The sensor is the camera which is configured to move and capture the scene. The obtaining unit is the acquisition unit. The image capturing apparatus is the estimation unit that estimates the camera’s position and orientation.).
Kotake does not explicitly disclose, however, Nakazato, in the same field of endeavor, teaches detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited (See at least paragraph [0043], “In reality, the relative positions and orientations of the light source and the camera are already defined by calibration. Accordingly, a correction amount corresponding to the position and direction of the measurement point is extracted from the correction amount database provided in advance, and the correction amount is added to the three-dimensional position of the measurement point to correct the depth information, and the corrected data is output. The method for creating the correction amount database is described with reference to the flowchart described below” and paragraph [0049], “In step S0020, a correction amount database is created. First, with respect to an arbitrary point on the surface of the measurement target object, using the depth information measured in step S0010, a difference between the positions of the point before and after the coating is acquired and registered as a correction amount in the database. Further, the position and the orientation of the measurement target object is registered in the correction amount database as a key for the database.”); correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point (See at least paragraph [0043], “In reality, the relative positions and orientations of the light source and the camera are already defined by calibration. Accordingly, a correction amount corresponding to the position and direction of the measurement point is extracted from the correction amount database provided in advance, and the correction amount is added to the three-dimensional position of the measurement point to correct the depth information, and the corrected data is output. The method for creating the correction amount database is described with reference to the flowchart described below” and paragraph [0049], “In step S0020, a correction amount database is created. First, with respect to an arbitrary point on the surface of the measurement target object, using the depth information measured in step S0010, a difference between the positions of the point before and after the coating is acquired and registered as a correction amount in the database. Further, the position and the orientation of the measurement target object is registered in the correction amount database as a key for the database.”); and correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points (See at least paragraph [0044], “In the extraction of the correction amount from the database, if positions and orientations of points are close, the correction values are close too. Consequently, if the position and the direction of a key point do not exist in the correction amount database, the correction amount can be calculated by interpolating from correction amounts of points having close positions and orientations. Alternatively, the position and orientation and the correction amount of the point may be put into a polynomial or a trigonometric function as parameters to generate a correction function for returning a correction amount using the position and orientation of the point as arguments, and the value may be used” and paragraph [0093], “Similarly to the first modification in the first exemplary embodiment, the processing in steps S3020, S3030, and S3040 can be repeatedly performed to correct the depth information. In such a case, with respect to the position of the measurement point corrected in step S3040 by the depth information correction unit 240, in step S3020, the position and orientation calculation unit 220 calculates the position and the orientation of the target object again, and in step S3030, the surface information acquisition unit 230 calculates the position and the direction of the measurement point. Then, based on the newly calculated position and direction of the measurement point, in step S3040, the depth information correction unit 240 corrects the depth information of the measurement point, that is, the three-dimensional position of the measurement point. The processing is repeated to correct the depth information until the change in the position and the orientation of the target object calculated by the position and orientation calculation unit 220 in step S3020 will be a value less than or equal to a predetermined threshold.” The system distinguishes between first applying a correction amount to directly affected points (local correction) and then applying interpolation/iterative corrections across a broader set of measurement points (global correction). The sequence corresponds to the claimed requirement that the second correction unit corrects a plurality of measurement points different from the third measurement points after the first correction is complete.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Kotake with the teachings of Nakazato such that the information processing apparatus of Kotake is further configured to utilize detecting, based on the output from the sensor, a second measurement point when a first measurement point is revisited, correcting, at a position where the revisit is detected, position and orientation information of one or more third measurement points used to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point, and correcting, after position and orientation information of the third measurement points have been corrected by the first correction, position and orientation information associated with a plurality of measurement points different from the third measurement points, as taught Nakazato (See paragraph [0043], [0044], [0049], [0093].), with a reasonable expectation of success. The motivation for doing so would be increased measurement accuracy, as taught by Nakazato (See paragraph [0012].).
Claim(s) 2, 3, 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kotake (US 20200012877 A1) in view of Nakazato (US 20120316820 A1) and HOU (US 20180005015 A1).
Regarding Claim 2, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake and Nakazato do not explicitly disclose, however, Hou, in the same field of endeavor, teaches wherein, in a case where a loop is detected by the detection unit, the generation unit does not generate a new measurement point until the correction by the first correction unit is completed (See at least paragraphs [0014], [0079], [0080], “The loop closure process in sparse SLAM helps dense SLAM to correct loops with few shape features but many color features” and “FIG. 17A depicts key frames without loop closure. As shown, there are significant drifting errors in the circle 1700. FIG. 17B depicts key frames after loop closure is completed. The drifting errors in circle 1700 no longer appear. Once the module 206 has completed the loop closure process, the module 206 updates the mapping database 208 with the latest inserted key frame.” The image processing module is the generation unit, which updates the map after loop correction with the key frame which is made up of measurement points.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Kotake with the teachings of Nakazato and HOU such that the information processing apparatus of Kotake is further configured to utilize a detection unit configured to detect, based on the output from the sensor, a second measurement point when a first measurement point is revisited, a first correction unit configured to correct, at a position where the revisit is detected, position and orientation information of one or more third measurement points used by the estimation unit to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point, and a second correction unit configured to correct, after position and orientation information of the third measurement points have been corrected by the first correction unit, position and orientation information associated with a plurality of measurement points different from the third measurement points, as taught Nakazato (See paragraph [0043], [0044], [0049], [0093].), and to withhold generating a new measurement point until correction is completed when a loop is detected, as taught by HOU (See paragraph [0079], [0080].), with a reasonable expectation of success. The motivation for doing so would be increased measurement accuracy, as taught by Nakazato (See paragraph [0012].). The motivation for doing so would be enhancing object reconstruction capability and improving speed and reliability of scene reconstruction, as taught by HOU (See paragraph [0005].).
Regarding Claim 3, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake and Nakazato do not explicitly disclose, however, Hou, in the same field of endeavor, teaches wherein the first correction unit corrects the position and orientation using rigid transformation (See at least paragraphs [0077], [0078], [0079], “FIG. 16A depicts the initial position of the latest inserted key frame 1602 and the initial position of the matched key frame 1604 from the mapping database 208 in the global coordinate system. As shown in FIG. 16A, the initial positions are quite far apart. FIG. 16B depicts the positions of the latest inserted key frame 1602 and the matched key frame 1604 after 3D rigid transformation occurs” and “Next, to close the loop (1406), the module 206 merges the latest inserted key frame with the matched key frame by merging the matched feature points and mapping points, and connects the key frames on one side of the loop to key frames on another side of the loop. The drifting error accumulated during the loop can be corrected through global bundle adjustment.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to have modified Kotake to incorporate the teachings of Nakazato and Hou for the same reasons stated in the motivation statement for claim 2.
Regarding Claim 13, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake and Nakazato do not explicitly disclose, however, Hou, in the same field of endeavor, teaches wherein the detection unit detects a loop in a case where an object observed based on the sensor information associated with the first measurement point is output from the sensor that has moved and is observed based on the sensor information acquired by the acquisition unit (See at least paragraphs [0059], [0060], [0075], “Loop Closing—the loop closing module corrects drifting errors contained in the data of the mapping database that is accumulated during tracking of the object” and “FIG. 3 is a flow diagram of a method 300 for determining the sensor pose and key frame insertion (e.g., the tracking module processing), using the system 200 of FIG. 2. The image processing module 206 receives color and depth frames as input from the sensor 203. The module 206 calculates (302) 2D features of the object (e.g., 202a) from the color frame and gets 3D information of the object 202a from the depth frame” and “In conjunction with the mapping module processing for inserting a new key frame into the mapping database 208, the image processing module 206 also performs loop closing processing to minimize drifting error in the key frames. FIG. 14 is a flow diagram of a method 1000 for closing the loop for key frames in the mapping database 208 (e.g., the loop closing module processing), using the system 200 of FIG. 2. The image processing module 206 receives the latest inserted key frame as input, and matches (1402) the latest inserted key frame to the key frames in the mapping database 208 to detect a loop and if any key frame in the mapping database 208 matches with the latest inserted key frame, the frames are processed to close the loop.” The image processing module is the detection unit and the acquisition unit.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to have modified Kotake to incorporate the teachings of Nakazato and Hou for the same reasons stated in the motivation statement for claim 2.
Regarding Claim 18, Kotake and Nakazato teach The information processing apparatus according to claim 1, as set forth in the obviousness rejection. Kotake and Nakazato do not explicitly disclose, however, Hou, in the same field of endeavor, teaches wherein the correction by the first correction unit is executed before the correction by the second correction unit is executed (See at least paragraphs [0059], [0060], [0069], [0071], “Loop Closing—the loop closing module corrects drifting errors contained in the data of the mapping database that is accumulated during tracking of the object” and “The module 206 projects every map point in neighboring key frames from the global coordinate system to the newly-inserted key frame and vice versa. Then, the projected map point searches for the map point with similar 2D features that is closest to its projected position in the newly-inserted key frame. Fusing similar map points naturally increases the connectivity between the newly-inserted key frame and its neighbor key frames. It benefits both tracking reliability and mapping, because more map points and key frames are involved in tracking and local bundle adjustment in mapping.” The loop closing module is the second correction unit. The image processing module is the first correction unit. The image processing module as the first correction unit projects points to newly-insert key frame while comparing for similar features improving reliability, the next step is the loop closing module as the second processing unit performing bundle adjustment.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to have modified Kotake to incorporate the teachings of Nakazato and Hou for the same reasons stated in the motivation statement for claim 2.
Claim(s) 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kotake (US 20200012877 A1) in view of Nakazato (US 20120316820 A1) and ARGAMAN (WO 2019222358 A1).
Regarding Claim 15, Kotake and Nakazato teach The information processing apparatus according to claim 14, as set forth in the obviousness rejection. Kotake and Nakazato do not explicitly disclose, however, ARGAMAN, in the same field of endeavor, teaches wherein the moving unit moves based on an operation by a user (See at least paragraphs [0120], [0121], [0159], Fig. 3A.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Kotake with the teachings of Nakazato and ARGAMAN such that the information processing apparatus of Kotake is further configured to utilize a detection unit configured to detect, based on the output from the sensor, a second measurement point when a first measurement point is revisited, a first correction unit configured to correct, at a position where the revisit is detected, position and orientation information of one or more third measurement points used by the estimation unit to estimate the position and orientation, by using a difference between position and orientation information of the second measurement point before aligning with the first measurement point, and position and orientation information of the second measurement point after aligning with the first measurement point, and a second correction unit configured to correct, after position and orientation information of the third measurement points have been corrected by the first correction unit, position and orientation information associated with a plurality of measurement points different from the third measurement points, as taught Nakazato (See paragraph [0043], [0044], [0049], [0093].), and to operate the moving unit based on user operation, as taught by ARGAMAN (See paragraph [0120], [0121].), with a reasonable expectation of success. The motivation for doing so would be increased measurement accuracy, as taught by Nakazato (See paragraph [0012].). The motivation for doing so would be enhancing navigational response and monitoring the moving unit, as taught by ARGAMAN (See paragraph [0004].).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEWEL ASHLEY KUNTZ whose telephone number is (571)270-5542. The examiner can normally be reached M-F 8:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached at (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEWEL ASHLEY KUNTZ/Examiner, Art Unit 3666
/ANNE MARIE ANTONUCCI/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights