DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The action is responsive to the following communication: an application filed on 07/24/2024 where:
Claims 1-11 are currently pending.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Allowable Subject Matter
Claims 3, 4, and 8 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 7, 9, 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over BALACHANDRESWARAN et al. (W0 2017/117675, hereinafter BALACHANDRESWARAN) in view of Pirchheim et al. (US 2014/0320593, hereinafter Pirchheim).
Regarding claim 1, BALACHANDRESWARAN teaches: An information processing device (fig. 1, a head mounted device ("HMD") 100) comprising:
a state information acquisition unit that extracts a feature point from each of latest frames of a moving image currently captured, and acquires state information regarding a position and a posture of a device equipped with a camera capturing the moving image, on a basis of a relation between a corresponding feature point of a previous frame to be collated and a point on a subject represented by these feature points ([0011], The camera system may perform SLAM at a tracking phase subsequent to the initialization phase by: (i) obtaining parameters of the front facing cameras; (ii) receiving the image stream corresponding to the image stream frames from at least one of the front facing cameras at a second epoch subsequent to a first epoch used in the initialization phase; (ii) extracting features from the image stream frames; (iii) generating a descriptor for each extracted feature; (iv) matching features between the image stream frames to those of the first epoch based on the descriptors).
BALACHANDRESWARAN does not explicitly teach: a registration information generation unit that determines frames included in the latest frames and meeting a predetermined condition as keyframes used as a reference for the previous frame, classifies each of the keyframes into any one of a plurality of groups for which different discard rules are set, registers the classified keyframes, and then discards, on a basis of the discard rules, any of pieces of data of keyframes previously registered; and
a registration information storage unit that stores data of each of the keyframes together with information indicating the classification.
However, Pirchheim teaches: a registration information generation unit that determines frames included in the latest frames and meeting a predetermined condition as keyframes used as a reference for the previous frame ([0032], HSLAM can insert a new keyframe into the 3D map if the current camera position (i.e., translation) is sufficiently far away from every existing keyframe position. HSLAM may also insert a new keyframe into the map if the current frame's coverage with known features is below a threshold (e.g., a new or previously unmapped region of the 3D map is represented in a current frame)), classifies each of the keyframes into any one of a plurality of groups for which different discard rules are set ([0005, 0060], Thus, HSLAM effectively converts panorama keyframes into regular 6DOF keyframes. Panorama keyframe or a regular keyframe. Different rule applies to them.), registers the classified keyframes, and then discards, on a basis of the discard rules, any of pieces of data of keyframes previously registered (fig. 7, [0051], Tracked 6DOF poses may be stored chronologically in memory. HSLAM can compute the parallax angle between the current pose and the stored poses and discard all poses with high parallax (e.g., greater than 5 degrees). 6DOF measurements 770 can move the system back to the full 6DOF mapping mode 755. If there is a tracking failure, relocalization 775 can recover a full 6DOF pose.); and
a registration information storage unit that stores data of each of the keyframes together with information indicating the classification ([0051, 0063], HSLAM can mark or tag the respective keyframe as a 6DOF keyframe. [0025], In one embodiment, HSLAM can perform 6DOF SLAM, which includes tracking and mapping of a Global SLAM Map as described above. HSLAM can maintain a single SLAM Map (i.e., the Global SLAM Map) and both 6DOF SLAM and Panorama SLAM can access and update the Global SLAM Map. SLAM).
Therefore, the Applicant's claimed invention would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of BALACHANDRESWARAN to include a registration information generation unit that determines frames included in the latest frames and meeting a predetermined condition as keyframes used as a reference for the previous frame, classifies each of the keyframes into any one of a plurality of groups for which different discard rules are set, registers the classified keyframes, and then discards, on a basis of the discard rules, any of pieces of data of keyframes previously registered; and a registration information storage unit that stores data of each of the keyframes together with information indicating the classification as taught by Pirchheim. The motivation/suggestion would have been to further enhance/improve the information processing device since doing so would allow for making more efficient choice of keyframes; therefore, reducing the overall computational cost for simultaneous localization and mapping tasks.
Regarding claim 5, BALACHANDRESWARAN and Pirchheim teach: The information processing device according to anyone in claim 1, wherein the registration information generation unit classifies the keyframes on a basis of at least any one of the number of extracted feature points, an acquisition time point, the number of times of use for collation, and a difference between a state of the device at a time of imaging and other states (BALACHANDRESWARAN, [0100], A second region 1 106 in the indicator 1102 shows the user which areas require more data to map. The user is thus prompted to complete one or more rotations within the real environment until the processor set has determined that the threshold amount of spatial awareness has been met. For example, the processor set may require the user to capture a complete 360 360-degree view of a room prior to rendering any virtual elements within the 3D map of the room.).
Regarding claim 7, BALACHANDRESWARAN and Pirchheim teach: The information processing device according to claim 1, wherein the state information acquisition unit acquires the state information on a basis of the moving image that indicates a space around a user wearing a head mounted display as the device and that is captured by movement of the user (BALACHANDRESWARAN, see fig. 1, a head mounted device ("HMD") 100).
Regarding claim 9, BALACHANDRESWARAN and Pirchheim teach The information processing device according to claim 7, further comprising: an application execution unit that forms a display image in a visual field corresponding to a visual line of the user on a basis of state information regarding the head mounted display, wherein the state information acquisition unit acquires the state information regarding the head mounted display by using data of the keyframes stored in the registration information storage unit before the application execution unit starts processing. (BALACHANDRESWARAN, see fig. 1, a head mounted device ("HMD") 100 and [0075], ).
Claims 10 and 11 are rejected for reasons similar to claim 1 above.
Claims 2 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over BALACHANDRESWARAN et al. (W0 2017/117675, hereinafter BALACHANDRESWARAN) in view of Pirchheim et al. (US 2014/0320593, hereinafter Pirchheim) and further in view of Li (US 2022/0398845).
Regarding claim 2, BALACHANDRESWARAN and Pirchheim teach: The information processing device according to claim 1.
BALACHANDRESWARAN and Pirchheim do not explicitly teach: wherein the registration information generation unit classifies some of the keyframes into a group prohibiting discard.
However, Li teaches: wherein the registration information generation unit classifies some of the keyframes into a group prohibiting discard ([0004], Further, a furthest keyframe is marginalized, and a newest set of frames and another set of keyframes are reserved.).
Therefore, the Applicant's claimed invention would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of BALACHANDRESWARAN and Pirchheim to include wherein the registration information generation unit classifies some of the keyframes into a group prohibiting discard as taught by Li. The motivation/suggestion would have been to further enhance/improve the information processing device since doing so would allow for making more efficient choice of keyframes; therefore, retaining essential keyframes while discarding redundant ones.
Regarding claim 6, BALACHANDRESWARAN and Pirchheim teach: The information processing device according to claim 1.
BALACHANDRESWARAN and Pirchheim do not explicitly teach: wherein the registration information generation unit discards the keyframe registered first in the keyframes previously registered and belonging to one of the groups at the time of an excess of a registration upper limit of the keyframes due to registration of the new keyframe.
However, Li teaches: wherein the registration information generation unit discards the keyframe registered first in the keyframes previously registered and belonging to one of the groups at the time of an excess of a registration upper limit of the keyframes due to registration of the new keyframe ([0004], Further, a furthest keyframe is marginalized, and a newest set of frames and another set of keyframes are reserved.).
Therefore, the Applicant's claimed invention would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of BALACHANDRESWARAN and Pirchheim to include wherein the registration information generation unit discards the keyframe registered first in the keyframes previously registered and belonging to one of the groups at the time of an excess of a registration upper limit of the keyframes due to registration of the new keyframe as taught by Li. The motivation/suggestion would have been to further enhance/improve the information processing device since doing so would allow for making more efficient choice of keyframes; therefore, retaining essential keyframes while discarding redundant ones.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW H LAM whose telephone number is (571)270-7969 and fax number is 571-270-8969. The examiner can normally be reached on 9AM-5PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Tieu can be reached on 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW H LAM/
Primary Examiner, Art Unit 2682
Prosecution Insights