DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of JP2022-109047, filed in Japan on 7/6/2022.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/27/2024 and 9/1/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6 and 10-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kato (U.S. Patent Pub. No. 2021/0042513) in view of Yitzhak (U.S. Patent No. 10838515).
Regarding Claim 1, Kato teaches information processing system comprising a head-mounted display and a controller, and a controller configured to estimate a position-orientation of the controller, wherein the head-mounted display includes (Fig. 2, 20 HMD, 10 Controller; ¶47 As illustrated in FIG. 2, the HMD 20 according to the present embodiment includes a control unit 200, the cameras 210, a posture sensor 220, an operation input unit 230, a communication unit 240, the display 250, the sound outputting unit 260, and a storage 270:)
a first camera; and one or more processors and/or circuitry configured to (Fig. 2; ¶47 As illustrated in FIG. 2, the HMD 20 according to the present embodiment includes a control unit 200, the cameras 210, a posture sensor 220, an operation input unit 230, a communication unit 240, the display 250, the sound outputting unit 260, and a storage 270)
perform first acquiring processing to acquire position-orientation information of the head-mounted display by using a captured image captured by the first camera (Fig. 6; ¶83 the HMD 20 acquires the peripheral images of the HMD 20 from the cameras 210 and the posture-sensor data from the posture sensor 220 (Step S121), and estimates, by using the environment map, a user position (namely, self-position of HMD 20) (Step S124);)
perform generating processing to generate map information based on the position-orientation information of the head-mounted display and a keyframe image that is a captured image captured at that position-orientation (Fig. 6; ¶the HMD 20 acquires (captures) peripheral images by using the cameras 210 of the HMD 20 (Step S103), and causes the environment mapping unit 201 to generate an environment map (Step S106). The generated environment map is stored in the environment-map storage 271)
the controller includes:
a second camera; and one or more processors and/or circuitry configured to (¶60 As illustrated in FIG. 2, the controller 10 according to the present embodiment includes a control unit 100, the cameras 110, a posture sensor 120, and a communication unit 130) perform second acquiring processing to acquire position-orientation information of the controller by using a captured image captured by the second camera and the (¶81 Next, the controller 10 acquires (captures) peripheral images by using the cameras 110 of the controller 10 (Step S109), and further executes sensing by using a posture sensor (Step S112); ¶82 Next, the controller 10 transmits, to the HMD 20, the peripheral images (captured images) and posture-sensor data (Step S115). Kato does not teach sending part of the map to the controller but rather taking the position orientation data determined by the controller and sending it to the HMD. Once the HMD receives this information it then uses the map along with the information to determine controller position see figs. 4 and 6)
Kato does not explicitly disclose perform extracting processing to extract a part of the map information; and perform communicating processing to transmit extracted map information extracted in the extracting processing to the controller.
Yitzhak is in the same field of art of image analysis. Further, Yitzhak teaches perform extracting processing to extract a part of the map information; and perform communicating processing to transmit extracted map information extracted in the extracting processing to the controller (Fig. 3; Col 8 Lines 23-37: the headset 105 selects 325 a portion of the map to send to the controller 110. One or more rules may be used when selecting which portion of the map to send to the controller 110, as discussed in FIG. 2. The headset 105 sends 330 the selected portion of the map to the controller 110. The controller 110 uses the selected portion of the map and the components of the controller 110 to determine its pose. The controller 110 may also use information received from an external base station or external tracking system to determine its pose. Additionally, the controller 110 may resolve features within the portion of the map, identify previously unseen features, and update scaling metrics. The headset 105 receives 335 pose information of the controller 110 from the controller 110.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kato by extracting part of the map and transmitting it to the controller that is taught by Yitzhak; thus, one of ordinary skilled in the art would be motivated to combine the references in order to not limit user mobility because users and the controllers must remain within the field of view of the external tracking device to accurately track the location of the headset and controllers (Yitzhak Background).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Kato in view of Yitzhak discloses the information processing system according to claim 1, wherein, in the second acquiring processing, the position-orientation information of the controller is acquired by using a captured image captured by the second camera, and at a predetermined timing, the position-orientation information of the controller is acquired by using a captured image captured by the second camera and the extracted map information transmitted from the head-mounted display (Kato, ¶78 The content-display controlling unit 204 generates an image in accordance with a real-time position and a real-time posture of the controller 10, which are estimated by the controller-position estimating unit 203, and thus a movement of the controller 10 in an actual space is reflected on a virtual space in real time to be able to further increase sense of immersion.)
Regarding Claim 3, Kato in view of Yitzhak discloses the information processing system according to claim 2, wherein the predetermined timing is a regular timing or a fixed timing (Kato, ¶78 The content-display controlling unit 204 generates an image in accordance with a real-time position and a real-time posture of the controller 10, which are estimated by the controller-position estimating unit 203, and thus a movement of the controller 10 in an actual space is reflected on a virtual space in real time to be able to further increase sense of immersion.)
Regarding Claim 4, Kato in view of Yitzhak discloses the information processing system according to claim 1, wherein the map information includes a plurality of pieces of keyframe information each associating the position-orientation information of the head-mounted display and the keyframe image captured at that position-orientation (Yitzhak, Col 6 Lines 10-12: The key feature module 205 detects key features in the environment and generates a map of an environment, as shown in FIG. 1B, using the one or more cameras…The map segmentation module 215 segments the map of the environment 100 generated by the key feature module 205. In some embodiments, the map segmentation module 215 determines which portion of the map should be sent to the display 230 and which portion of the map should be sent to the controller 110.)
The reasons for combining Kato and Yitzhak are similar to that stated in the rejection of claim 1. In addition, this same reasoning is pertinent and applicable to the rejections of claims 5-6, 10-12, and 14-18 below.
Regarding Claim 5 Kato in view of Yitzhak discloses the information processing system according to claim 4, wherein, in the extracting processing, the keyframe information having been created at a nearby area around the controller is extracted from the map information (Yitzhak, Col 6 Lines 42-47: To determine which portion of the map should be sent to the display 230, the map segmentation module 215 obtains location information of the headset 105 from the tracking module 210, location information of the controller 110, and information from tracking the user's movement within the environment 100.)
Regarding Claim 6, Kato in view of Yitzhak discloses the information processing system according to claim 5, wherein a size of the nearby area is determined based on at least any of a processing capacity of the controller, a storage capacity of a storage portion included in the controller, and number of pieces of the keyframe information within the nearby area (It would be obvious to one with ordinary skill in the art to determine the size which needs to be transmitted for better computing efficiency; Yitzhak Col 6 Lines 48-56: To determine which portion of the map should be sent to the controller 110, the map segmentation module 215 uses the controller's 110 estimated location from the tracking module 210 and compares it to a set of rules. Examples of rules include selecting a portion of the map that is within threshold distance between the estimated location of the controller 110, selecting a portion of the map that is within a threshold field of view of the controller 110, sending a pre-determined fraction of the map to the controller 110, etc.)
Regarding Claim 10, Kato in view of Yitzhak discloses the information processing system according to claim 4, wherein, in the extracting processing, the keyframe information is extracted based on variance in positions of a feature point captured in the keyframe image or a reprojection error of the feature point (Yitzhak, Col 7 Lines 44-55: the key feature module 240 may add objects to the portion of the map that were not previously seen by the headset 105. This may include adding objects that were not in the environment when the headset 105 generated the map of the environment. Similarly, objects may be added that were not in the environment when the headset 105 sent the portion of the map to the controller 110. The key feature module 240 may also resolve features (e.g., curves, contours, edges) in objects in the portion of the map that were distorted or miscalculated. Such distortions and miscalculations may occur because of headset's elevated point of view, limited range of motion, and large form factor.)
Regarding Claim 11, Kato in view of Yitzhak discloses the information processing system according to claim 4, wherein, in the extracting processing, the keyframe information is extracted from the map information, based on a ratio by which the keyframe image is occupied by blur, number of moving objects included in the keyframe image, or timing at which the keyframe information is generated (Yitzhak, Col 7 Lines 44-55: the key feature module 240 may add objects to the portion of the map that were not previously seen by the headset 105. This may include adding objects that were not in the environment when the headset 105 generated the map of the environment. Similarly, objects may be added that were not in the environment when the headset 105 sent the portion of the map to the controller 110. The key feature module 240 may also resolve features (e.g., curves, contours, edges) in objects in the portion of the map that were distorted or miscalculated. Such distortions and miscalculations may occur because of headset's elevated point of view, limited range of motion, and large form factor.)
Regarding Claim 12, Kato in view of Yitzhak discloses the information processing system according to claim 4, wherein, in the extracting processing, the keyframe information is extracted from the map information, based on a direction in which the controller captures an image (Yitzhak, Col 7-8 Lines 61-6: As a result, the controller 110 can resolve features of objects that were incorrectly seen by the controller 110. For example, the controller 110 can resolve the table as a three-dimensional object. The controller 110 would then update the portion of the map to include the correct dimensions of the table. The key feature module 240 can also update scaling metrics miscalculated by the headset 105 for the portion of the map. The updated portion of the map is stored in the map store 255. The key feature module 240 also sends the updated portion of the map to the headset 105. In some embodiments, the controller 110 sends the entire updated portion of the map to the headset 105.)
Regarding Claim 13, Kato in view of Yitzhak discloses the information processing system according to claim 1, wherein
the controller further includes an inertial measurement unit, and (Kato, Fig. 2, ¶63 the posture sensor 120 is realized by an inertial measurement unit (IMU))
in the second acquiring processing, the position-orientation information of the controller is acquired by using the inertial measurement unit, and at a predetermined timing, the position-orientation information of the controller is acquired by using a captured image captured by the second camera and the extracted map information transmitted from the head-mounted display (Kato, ¶81 Next, the controller 10 acquires (captures) peripheral images by using the cameras 110 of the controller 10 (Step S109), and further executes sensing by using a posture sensor (Step S112); ¶82 Next, the controller 10 transmits, to the HMD 20, the peripheral images (captured images) and posture-sensor data (Step S115). Kato does not teach sending part of the map to the controller but rather taking the position orientation data determined by the controller and sending it to the HMD. Once the HMD receives this information it then uses the map along with the information to determine controller position see figs. 4 and 6; ¶78 The content-display controlling unit 204 generates an image in accordance with a real-time position and a real-time posture of the controller 10, which are estimated by the controller-position estimating unit 203, and thus a movement of the controller 10 in an actual space is reflected on a virtual space in real time to be able to further increase sense of immersion)
Regarding Claim 14, Kato in view of Yitzhak discloses the information processing system according to claim 1, wherein, in the extracting processing, an amount to be extracted or a ratio to be extracted from the map information is changed based on at least any of a processing capacity and a storage capacity of the controller (It would be obvious to one with ordinary skill in the art to determine the size which needs to be transmitted for better computing efficiency; Yitzhak Col 6 Lines 48-56: To determine which portion of the map should be sent to the controller 110, the map segmentation module 215 uses the controller's 110 estimated location from the tracking module 210 and compares it to a set of rules. Examples of rules include selecting a portion of the map that is within threshold distance between the estimated location of the controller 110, selecting a portion of the map that is within a threshold field of view of the controller 110, sending a pre-determined fraction of the map to the controller 110, etc.)
Regarding Claim 15, Kato in view of Yitzhak discloses the information processing system according to claim 1, wherein, in the extracting processing, an amount to be extracted or a ratio to be extracted from the map information is changed based on a speed of movement of the controller or an acquisition condition of the position-orientation information of the controller (It would be obvious to one with ordinary skill in the art to determine the size which needs to be transmitted for better computing efficiency; Yitzhak Col 6 Lines 48-56: To determine which portion of the map should be sent to the controller 110, the map segmentation module 215 uses the controller's 110 estimated location from the tracking module 210 and compares it to a set of rules. Examples of rules include selecting a portion of the map that is within threshold distance between the estimated location of the controller 110 (this can be interpreted as speed), selecting a portion of the map that is within a threshold field of view of the controller 110, sending a pre-determined fraction of the map to the controller 110, etc.)
Regarding Claim 16, Kato in view of Yitzhak discloses the information processing system according to claim 1, wherein
in the extracting processing, pieces of the extracted map information are generated for a plurality of the controllers respectively, and in the communicating processing, the pieces of the extracted map information are transmitted to the plurality of the controllers respectively (It would be obvious to one skilled in the art to use multiple controllers if needed; Yitzhak, Col 1 Lines 51-53: Additional controllers, an external base station, an external tracking system, and other components may be included in the VR system.)
Regarding claim 17, claim 17 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Kato further teaching on: A head-mounted display generating map information for allowing a controller to estimate a position-orientation, the head-mounted display comprising:
a camera; and one or more processors and/or circuitry configured to (Kato, Fig. 2, 20 HMD, 10 Controller; ¶47 As illustrated in FIG. 2, the HMD 20 according to the present embodiment includes a control unit 200, the cameras 210, a posture sensor 220, an operation input unit 230, a communication unit 240, the display 250, the sound outputting unit 260, and a storage 270:)
Regarding claim 18, claim 18 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Kato further teaching on: A controller estimating a position-orientation using map information received from a head-mounted display, the controller comprising: a camera; and one or more processors and/or circuitry (Kato, Fig. 2, 20 HMD, 10 Controller; ¶47 As illustrated in FIG. 2, the HMD 20 according to the present embodiment includes a control unit 200, the cameras 210, a posture sensor 220, an operation input unit 230, a communication unit 240, the display 250, the sound outputting unit 260, and a storage 270:)
Regarding claim 19, claim 19 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Kato further teaching on: A method (Kato, ¶8 an information processing method, and a program capable of performing position estimation of a controller corresponding to a wearable device)
Allowable Subject Matter
Claims 7-9 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 7, no prior art teaches the information processing system according to claim 4, wherein, in the extracting processing, an area, where positions at which keyframe images are captured are distributed, is divided into a plurality of split areas, and a representative piece of the keyframe information is extracted from each of the split areas.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
Prosecution Insights