DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Summary of Response
2. The 09/02/2025 response includes: (a) claims 21-22 are new; (b) claims 1, 4, 11, 14-16 and 20 are currently amended; (c) claims 2-3, 5-7, 9-10, 12-13, 17 and 19 are original; (d) claims 8 and 18 are canceled; and (e) the grounds for rejection set forth in the 06/02/2025 office action are traversed. Claims 1-7, 9-17 and 19-22 are currently pending and an office action follows:
Response to Arguments
3. Applicant’s arguments filed 09/0/2025 with respect to the rejection of claims 1-7, 9-17 and 19-22 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, new grounds of rejection of claims 1-7, 9-17 and 19-22 is made in view of U.S. Patent Pub. No. 2020/0357102 A1 to Pekkucuksen et al. combined with previously-cited prior art. Applicant’s remarks appear to be moot in view of this new grounds of rejection.
Claim Objections
4. Claims 14 and 21-22 are objected to because of the following informalities:
Claim 14 at line 5 includes “the determining an estimated pose of the entity controller” should be changed to “the determining the estimated pose of the entity controller” because claim 11 already includes “determining an estimated pose of the entity controller. Appropriate correction is required.
Claim 21 at line 1 includes “the inertial data” that is lacks antecedent basis. This objection may be overcome, for example, by amending it to “inertial data”. Appropriate correction is required.
Claim 22 at line 1 includes “The computer-readable storage medium of claim 20” that should be changed to “The non-transitory computer-readable storage medium of claim 20” to be consistent. Appropriate correction is required.
Claim Rejections – 35 USC §112
5. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
6. Claims 9-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
More particularly, claims 9-10 and 19 respectively depend upon cancelled claims 8 and 18. This rejection may be overcome, for example, by amending claim 9 to “The method of claim 1” and amending claim 19 to “The electronic device of claim 11”. Appropriate correction is required.
Claim Rejections – 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1-3, 11-13, 20 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Pub. No. 2010/0105475 A1 to Mikhailov et al. (“Mikhailov”) in view of U.S. Patent Pub. No. 2020/0357102 A1 to Pekkucuksen et al. (“Pekkucuksen”).
PNG
media_image1.png
200
400
media_image1.png
Greyscale
PNG
media_image2.png
4555
3371
media_image2.png
Greyscale
PNG
media_image3.png
200
400
media_image3.png
Greyscale
PNG
media_image4.png
200
400
media_image4.png
Greyscale
PNG
media_image5.png
200
400
media_image5.png
Greyscale
As to claim 1, Mikhailov discloses a method for positioning an entity controller(C1/470/570)(Figs. 5, 18, 20; Abstract; ¶¶0043, 0055, 0091, 0099), comprising:
determining a relative pose relationship between the entity controller(C1/470/570)(Figs. 5, 18, 20; Abstract; ¶¶0055, 0091, 0099) and an associated part of the entity controller(part of C1/470/570)(Figs. 5, 9, 15A, 18, 20; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090, 0099, 0101), the relative pose relationship being determined from at least one entity controller pose and at least one associated part pose (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion” – as shown in FIG. 9 several images are used to determine the controller’s location, orientation and trajectory. Also, if an occluded/partial image/part pose of controller is detected then the controller’s position is based on dead reckoning that uses the controller’s position is estimated from a previous entity controller pose/full pose of controller along with its speed, time and trajectory), the entity controller pose being determined from a first type of image (Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose), the associated part pose being determined from a second image (Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose),
determining a current associated part pose, the current associated part pose being determined based on a current image that is closest to the current time (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”);
determining an estimated pose of the entity controller(C1/470/570)(Figs. 5-6, 18, 20; ¶¶0043, 0055-0057, 0091, 0099-0101) based on the relative pose relationship and the current associated part pose (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”).
Mikhailov does not expressly disclose the associated part pose being determined from a second type of image, and the first type of image and the second type of image using different exposure policies, wherein the first type of image and the second type of image are alternately captured at a predetermined frequency; the current associated part pose being determined based on a current second type of image that is closest to the current time.
Pekkucuksen discloses a second type of image, and the first type of image and the second type of image using different exposure policies (¶¶0098, especially – “The capture settings for a video sequence can be alternated to provide image frames with different exposure times, and a high-frame rate capture capability”, 0101), wherein the first type of image and the second type of image are alternately captured at a predetermined frequency (¶¶0098, especially – “The capture settings for a video sequence can be alternated to provide image frames with different exposure times, and a high-frame rate capture capability”, 0101).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Mikhailov with Pekkucuksen to provide a method for positioning an entity controller that is better able to detect the position of the entity controller (i.e., by creating an image with less motion blur and less noise, ¶¶0032-0033).
Mikhailov and Pekkucuksen teach the associated part pose being determined from a second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose; Pekkucuksen: ¶¶0098, 0101), and the first type of image and the second type of image using different exposure policies (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101), wherein the first type of image and the second type of image are alternately captured at a predetermined frequency (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101); the current associated part pose being determined based on a current second type of image that is closest to the current time (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”; Pekkucuksen: ¶¶0098, 0101).
As to claim 2, Mikhailov and Pekkucuksen teach the method of claim 1, as applied above.
Mikhailov and Pekkucuksen further teach wherein the entity controller is provided with a light-emitting element (Mikhailov: Figs. 5, 18, 20: C1/470/570; Abstract; ¶¶0045, 0055, 0089-0091, 0099), the first type of image comprises a light spot formed by the light-emitting element (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0089-0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101), and the entity controller pose is determined from the light spot in the first type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0089-0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
As to claim 3, Mikhailov and Pekkucuksen teach the method of claim 1, as applied above.
Mikhailov and Pekkucuksen further teach wherein the determining an estimated pose of the entity controller (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0099-0101) based on the relative pose relationship and the current associated part pose (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”) comprises:
determining the estimated pose of the entity controller based on the relative pose relationship, the current associated part pose and a first confidence, and a previous estimated pose of the entity controller, inertial data and a second confidence (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0065, 0066, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position. Also, this estimate is based on the images each being given a weight/confidence that decreases as the image ages and therefore first and second confidence values may correspond to any two of the latest image taken and the multiple previous images taken directly before it; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
PNG
media_image6.png
200
400
media_image6.png
Greyscale
As to claim 11, Mikhailov discloses an electronic device(1400)(FIG. 22; ¶0103), comprising:
a processor(1428)(FIG. 22; ¶0103); and
a memory(1426)(FIG. 22; ¶¶0103, 0130-0131) for storing executable instructions for the processor(1428)(FIG. 22; ¶¶0103, 0130-0131);
wherein the processor(1428)(FIG. 22; ¶¶0103, 0130-0131) is configured to perform, by executing the executable instructions (FIG. 22: 1428; ¶¶0103, 0130-0131), a method (Figs. 5, 18, 20: C1/470/570; Abstract; ¶¶0043, 0055, 0091, 0099) comprising:
determining a relative pose relationship between an entity controller(C1/470/570)(Figs. 5, 18, 20; Abstract; ¶¶0055, 0091, 0099) and an associated part of the entity controller(part of C1/470/570)(Figs. 5, 9, 15A, 18, 20; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090, 0099, 0101), the relative pose relationship being determined from at least one entity controller pose and at least one associated part pose (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion” – as shown in FIG. 9 several images are used to determine the controller’s location, orientation and trajectory. Also, if an occluded/partial image/part pose of controller is detected then the controller’s position is based on dead reckoning that uses the controller’s position is estimated from a previous entity controller pose/full pose of controller along with its speed, time and trajectory), the entity controller pose being determined from a first type of image (Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose), the associated part pose being determined from a second image (Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose);
determining a current associated part pose, the current associated part pose being determined based on a current image that is closest to a current time (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”);
determining an estimated pose of the entity controller(C1/470/570)(Figs. 5, 18, 20; ¶¶0043, 0055, 0091, 0099) based on the relative pose relationship and the current associated part pose (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”).
Mikhailov does not expressly disclose the associated part pose being determined from a second type of image, and the first type of image and the second type of image using different exposure policies, wherein the first type of image and the second type of image are alternately captured at a predetermined frequency; the current associated part pose being determined based on a current second type of image that is closest to the current time.
Pekkucuksen discloses a second type of image, and the first type of image and the second type of image using different exposure policies (¶¶0098, especially – “The capture settings for a video sequence can be alternated to provide image frames with different exposure times, and a high-frame rate capture capability”, 0101), wherein the first type of image and the second type of image are alternately captured at a predetermined frequency (¶¶0098, especially – “The capture settings for a video sequence can be alternated to provide image frames with different exposure times, and a high-frame rate capture capability”, 0101).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Mikhailov with Pekkucuksen to provide an electronic device that is better able to detect the position of the entity controller (i.e., by creating an image with less motion blur and less noise, ¶¶0032-0033).
Mikhailov and Pekkucuksen teach the associated part pose being determined from a second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose; Pekkucuksen: ¶¶0098, 0101), and the first type of image and the second type of image using different exposure policies (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101), wherein the first type of image and the second type of image are alternately captured at a predetermined frequency (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101); the current associated part pose being determined based on a current second type of image that is closest to the current time (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”; Pekkucuksen: ¶¶0098, 0101).
As to claim 12, Mikhailov and Pekkucuksen teach the electronic device of claim 11, as applied above.
Mikhailov and Pekkucuksen further teach wherein the entity controller is provided with a light-emitting element (Mikhailov: Figs. 5, 18, 20: C1/470/570; Abstract; ¶¶0045, 0055, 0089-0091, 0099), the first type of image comprises a light spot formed by the light-emitting element (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0089-0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101), and the entity controller pose is determined from the light spot in the first type of image (Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0089-0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
As to claim 13, Mikhailov and Pekkucuksen teach the electronic device of claim 11, as applied above.
Mikhailov and Pekkucuksen further teach wherein the determining an estimated pose of the entity controller (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0099-0101) based on the relative pose relationship and the current associated part pose (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”) comprises: determining the estimated pose of the entity controller based on the relative pose relationship, the current associated part pose and a first confidence, and a previous estimated pose of the entity controller, inertial data and a second confidence (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0065, 0066, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position. Also, this estimate is based on the images each being given a weight/confidence that decreases as the image ages and therefore first and second confidence values may correspond to any two of the latest image taken and the multiple previous images taken directly before it; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
As to claim 20, Mikhailov discloses a non-transitory computer-readable storage medium(1426)(FIG. 22; ¶¶0103, 0130-0131) having a computer program stored thereon (FIG. 22: 1426; ¶¶0103, 0130-0131), wherein the computer program (FIG. 22: 1426; ¶¶0103, 0130-0131), when executed by a processor(1428)(FIG. 22; ¶¶0103, 0130-0131), implements a method (Figs. 5, 18, 20: C1/470/570; Abstract; ¶¶0043, 0055, 0091, 0099) comprising:
determining a relative pose relationship between an entity controller(C1/470/570)(Figs. 5, 18, 20; Abstract; ¶¶0055, 0091, 0099) and an associated part of the entity controller(part of C1/470/570)(Figs. 5, 9, 15A, 18, 20; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090, 0099, 0101), the relative pose relationship being determined from at least one entity controller pose and at least one associated part pose (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion” – as shown in FIG. 9 several images are used to determine the controller’s location, orientation and trajectory. Also, if an occluded/partial image/part pose of controller is detected then the controller’s position is based on dead reckoning that uses the controller’s position is estimated from a previous entity controller pose/full pose of controller along with its speed, time and trajectory), the entity controller pose being determined from a first type of image (Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose), the associated part pose being determined from a second image(Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose);
determining a current associated part pose, the current associated part pose being determined based on a current image that is closest to the current associated part pose (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”);
determining an estimated pose of the entity controller(C1/470/570)(Figs. 5, 18, 20; ¶¶0043, 0055, 0091, 0099) based on the relative pose relationship and the current associated part pose (Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”).
Mikhailov does not expressly disclose the associated part pose being determined from a second type of image, and the first type of image and the second type of image using different exposure policies, wherein the first type of image and the second type of image are alternately captured at a predetermined frequency; the current associated part pose being determined based on a current second type of image that is closest to the current time.
Pekkucuksen discloses a second type of image, and the first type of image and the second type of image using different exposure policies (¶¶0098, especially – “The capture settings for a video sequence can be alternated to provide image frames with different exposure times, and a high-frame rate capture capability”, 0101), wherein the first type of image and the second type of image are alternately captured at a predetermined frequency (¶¶0098, especially – “The capture settings for a video sequence can be alternated to provide image frames with different exposure times, and a high-frame rate capture capability”, 0101).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Mikhailov with Pekkucuksen to provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a method that is better able to detect the position of the entity controller (i.e., by creating an image with less motion blur and less noise, ¶¶0032-0033).
Mikhailov and Pekkucuksen teach the associated part pose being determined from a second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose; Pekkucuksen: ¶¶0098, 0101), and the first type of image and the second type of image using different exposure policies (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101); the current associated part pose being determined based on a current second type of image that is closest to the current associated part pose (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”; Pekkucuksen: ¶¶0098, 0101).
As to claim 22, Mikhailov and Pekkucuksen teach the computer-readable storage medium of claim 20, as applied above.
Mikhailov and Pekkucuksen further teach wherein the entity controller is provided with a light-emitting element (Mikhailov: Figs. 5, 18, 20: C1/470/570; Abstract; ¶¶0045, 0055, 0089-0091, 0099), the first type of image comprises a light spot formed by the light-emitting element (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0089-0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101), and the entity controller pose is determined from the light spot in the first type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0089-0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 20.
9. Claims 4-7, 9-10, 14-17, 19 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Pub. No. 2010/0105475 A1 to Mikhailov et al. (“Mikhailov”) in view of U.S. Patent Pub. No. 2020/0357102 A1 to Pekkucuksen et al. (“Pekkucuksen”) as applied to claims 1, 9 and 11 above, in view of U.S. Patent Pub. No. 2023/0418370 A1 to Wu.
As to claim 4, Mikhailov and Pekkucuksen teach the method of claim 1, as applied above.
Mikhailov and Pekkucuksen teach further comprises: determining a current entity controller pose, the current entity controller pose being determined based on a current first type of image that is closest to a current time (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”; Pekkucuksen: ¶¶0098, 0101);
the determining an estimated pose of the entity controller based on the relative pose relationship and the current associated part pose (Mikhailov: Figs. 5-6, 18, 20: part of C1/470/570, C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101) comprises:
determining the estimated pose of the entity controller based on the relative pose relationship and the current associated part pose, the current entity controller pose, and inertial data corresponding to the current first type of image and corresponding to the current second type of image (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of the current first type of image and a capture time of the current second type of image.
Wu discloses and inertial data between a capture time of the current first image and a capture time of the current second image (¶0024).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Mikhailov and Pekkucuksen with Wu to provide a method for positioning an entity controller that very accurately determines the position and orientation of the entity controller (i.e., by interpolating image data at a higher rate according to the inertial data ¶0024).
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of the current first image and a capture time of the current second image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 5, Mikhailov and Pekkucuksen teach the method of claim 1, as applied above.
Mikhailov and Pekkucuksen further teach further comprises: determining a real-time pose of the entity controller (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0043, 0055, 0091, 0099-0101) based on the estimated pose of the entity controller and inertial data corresponding to a capture time of the current second type of image and corresponding to a current time (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of the current second type of image and a current time.
Wu discloses and inertial data between a capture time of the image and a current time (¶0024).
The motivation to combine Wu is set forth above for claim 4.
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of the image and a current time (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 6, Mikhailov and Pekkucuksen teach the method of claim 1, as applied above.
Mikhailov and Pekkucuksen further teach wherein the associated part pose is determined based on an entity region of interest in the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose; Pekkucuksen: ¶¶0098, 0101), and the entity region of interest in the second type of image is determined based on an entity controller pose corresponding to a previous first type of image of the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0067-0069, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose; Pekkucuksen: ¶¶0098, 0101) and inertial data corresponding to a capture time of the previous first type of image and corresponding to a capture time of the second type of image (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of the previous first type of image and a capture time of the second type of image.
Wu discloses and inertial data between a capture time of the previous first image and a capture time of the second image (¶0024).
The motivation to combine Wu is set forth above for claim 4.
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of the previous first type of image and a capture time of the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 7, Mikhailov and Pekkucuksen teach the method of claim 1, as applied above.
Mikhailov and Pekkucuksen further teach wherein the relative pose relationship is determined from at least one entity controller pose (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion” – as shown in FIG. 9 several images are used to determine the controller’s location, orientation and trajectory. Also, if an occluded/partial image/part pose of controller is detected then the controller’s position is based on dead reckoning that uses the controller’s position is estimated from a previous entity controller pose/full pose of controller along with its speed, time and trajectory), at least one associated part pose (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”), and inertial data corresponding to a capture time of a first type of image corresponding to the entity controller pose and corresponding to a capture time of a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose.
Wu discloses and inertial data between a capture time of a first image and a capture time of a second image (¶0024).
The motivation to combine Wu is set forth above for claim 4.
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 9, Mikhailov and Pekkucuksen teach the method of claim 8, as applied above.
Mikhailov and Pekkucuksen further teach wherein the relative pose relationship is determined from at least one entity controller pose, at least one associated part pose, and inertial data corresponding to a capture time of a first type of image corresponding to the entity controller pose and a capture time corresponding to a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101), and the capture time of the first type of image corresponding to the entity controller pose and the capture time of the second type of image corresponding to the associated part pose are adjacent (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion” – as shown in FIG. 9 several images are used to determine the controller’s location, orientation and trajectory. Also, if an occluded/partial image/part pose of controller is detected then the controller’s position is based on dead reckoning that uses the controller’s position is estimated from a previous entity controller pose/full pose of controller along with its speed, time and trajectory; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose.
Wu discloses and inertial data between a capture time of a first image and a capture time of a second image (¶0024).
The motivation to combine Wu is set forth above for claim 4.
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 10, Mikhailov and Pekkucuksen teach the method of claim 9, as applied above.
Mikhailov further discloses wherein the inertial data are inertial data of the entity controller (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0043, 0055, 0091, 0094, 0096-0097, 0099), and a capture frequency of the inertial data (Mikhailov: 0094, 0096-0097); a capture frequency of the first type of image and / or the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 1.
Mikhailov and Pekkucuksen do not expressly disclose and a capture frequency of the inertial data is higher than a capture frequency of the first type of image and / or the second type of image.
Wu discloses and a capture frequency of the inertial data is higher than a capture frequency of the image (¶0024).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Mikhailov and Pekkucuksen with Wu to provide a method for positioning an entity controller that: (i) uses a common inertial data measuring device; and (ii) very accurately determines the position and orientation of the entity controller (i.e., by interpolating image data at a higher rate according to the inertial data ¶0024).
Mikhailov, Pekkucuksen and Wu teach and a capture frequency of the inertial data is higher than a capture frequency of the first type of image and / or the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 14, Mikhailov and Pekkucuksen teach the electronic device of claim 11, as applied above.
Mikhailov and Pekkucuksen teach wherein the method further comprises: determining a current entity controller pose, the current entity controller pose being determined based on a current first type of image that is closest to a current time (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”; Pekkucuksen: ¶¶0098, 0101); the determining an estimated pose of the entity controller based on the relative pose relationship and the current associated part pose (Mikhailov: Figs. 5-6, 18, 20: part of C1/470/570, C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101) comprises: determining the estimated pose of the entity controller based on the relative pose relationship and the current associated part pose, the current entity controller pose, and inertial data corresponding to the current first type of image and corresponding to the current second type of image (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of the current first type of image and a capture time of the current second type of image.
Wu discloses and inertial data between a capture time of the current first image and a capture time of the current second image (¶0024).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Mikhailov and Pekkucuksen with Wu to provide an electronic device that very accurately determines the position and orientation of the entity controller (i.e., by interpolating image data at a higher rate according to the inertial data ¶0024).
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of the current first image and a capture time of the current second image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 15, Mikhailov and Pekkucuksen teach the electronic device of claim 11, as applied above.
Mikhailov and Pekkucuksen further teach wherein the method further comprises: determining a real-time pose of the entity controller (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0043, 0055, 0091, 0099-0101) based on the estimated pose of the entity controller and inertial data corresponding to a capture time of the current second type of image and corresponding to a current time (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of a current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of the current type of image and a current time.
Wu discloses and inertial data between a capture time of the image and a current time (¶0024).
The motivation to combine Wu is set forth above for claim 14.
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of the current type of image and a current time (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of a current position; Pekkucuksen: ¶¶0098, 0101; col 14, ln 52-67; Wu: ¶0024).
As to claim 16, Mikhailov and Pekkucuksen teach the electronic device of claim 11, as applied above.
Mikhailov and Pekkucuksen further teach wherein the associated part pose is determined based on an entity region of interest in the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose; Pekkucuksen: ¶¶0098, 0101), and the entity region of interest in the second type of image is determined based on an entity controller pose corresponding to a previous first type of image of the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0067-0069, 0091, 0099 – FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose; Pekkucuksen: ¶¶0098, 0101) and inertial data corresponding to a capture time of the previous first type of image and corresponding to a capture time of the second type of image (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of the previous first type of image and a capture time of the second type of image.
Wu discloses and inertial data between a capture time of the previous first image and a capture time of the second image (¶0024).
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of the previous first type of image and a capture time of the second type of image (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 17, Mikhailov and Pekkucuksen teach the electronic device of claim 11, as applied above.
Mikhailov and Pekkucuksen further teach wherein the relative pose relationship is determined from at least one entity controller pose (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion” – as shown in FIG. 9 several images are used to determine the controller’s location, orientation and trajectory. Also, if an occluded/partial image/part pose of controller is detected then the controller’s position is based on dead reckoning that uses the controller’s position is estimated from a previous entity controller pose/full pose of controller along with its speed, time and trajectory), at least one associated part pose (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion”), and inertial data corresponding to a capture time of a first type of image corresponding to the entity controller pose and corresponding to a capture time of a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose.
Wu discloses and inertial data between a capture time of a first image and a capture time of a second image (¶0024).
The motivation to combine Wu is set forth above for claim 14.
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 19, Mikhailov and Pekkucuksen teach the electronic device of claim 18, as applied above.
Mikhailov and Pekkucuksen further teach wherein the relative pose relationship is determined from at least one entity controller pose, at least one associated part pose, and inertial data corresponding to a capture time of a first type of image corresponding to the entity controller pose and corresponding to a capture time of a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101), and the capture time of the first type of image corresponding to the entity controller pose and the capture time of the second type of image corresponding to the associated part pose are adjacent (Mikhailov: Figs. 5, 9, 15A, 18, 20: relative pose relationship between C1/470/570 and part of; Abstract, especially – “A ball section is attached to the game controller to locate the controller using visual information”; ¶¶0043, 0045, 0055-0056, 0057, especially – “if the ball gets occluded due to the movement of the controller, only a partial image may be captured by the camera”, 0060, especially – “only a partial section of the ball has been captured”, 0065-0066, 0082, especially – “only about half ring 352 has been detected, such as when the ball is occluded”, 0090-0091, 0099, 0101, especially – “dead reckoning can be combined at times with visual tracking…because of partial occlusion” – as shown in FIG. 9 several images are used to determine the controller’s location, orientation and trajectory. Also, if an occluded/partial image/part pose of controller is detected then the controller’s position is based on dead reckoning that uses the controller’s position is estimated from a previous entity controller pose/full pose of controller along with its speed, time and trajectory; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
Mikhailov and Pekkucuksen do not expressly disclose and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose.
Wu discloses and inertial data between a capture time of a first image and a capture time of a second image (¶0024).
The motivation to combine Wu is set forth above for claim 14.
Mikhailov, Pekkucuksen and Wu teach and inertial data between a capture time of a first type of image corresponding to the entity controller pose and a capture time of a second type of image corresponding to the associated part pose (Mikhailov: Figs. 5-6, 18, 20: C1/470/570; ¶¶0043, 0055-0057, 0091, 0094, 0096-0097, 0099-0101 - FIG. 20 shows an occluded image/part pose to the right of unoccluded image/entity controller pose that is estimated {FIG. 6} using this image and a previous unoccluded image along with inertial data {e.g., inertial acceleration data} as part of dead reckoning’s determination of the position corresponding to a second image as well as a later/current position; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
As to claim 21, Mikhailov and Pekkucuksen teach the electronic device of claim 11, as applied above.
Mikhailov and Pekkucuksen further teach wherein the inertial data are inertial data of the entity controller (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0043, 0055, 0091, 0094, 0096-0097, 0099), and a capture frequency of the inertial data (Mikhailov: 0094, 0096-0097); a capture frequency of the first type of image and/or the second type of image data (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101).
The motivation to combine Pekkucuksen is set forth above for claim 11.
Mikhailov and Pekkucuksen do not expressly disclose and a capture frequency of the inertial data is higher than a capture frequency of the first type of image and/or the second type of image data.
Wu discloses and a capture frequency of the inertial data is higher than a capture frequency of the image (¶0024).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Mikhailov and Pekkucuksen with Wu to provide an electronic device that: (i) uses a common inertial data measuring device; and (ii) very accurately determines the position and orientation of the entity controller (i.e., by interpolating image data at a higher rate according to the inertial data ¶0024).
Mikhailov, Pekkucuksen and Wu teach and a capture frequency of the inertial data is higher than a capture frequency of the first type of image and / or the second type of image (Mikhailov: Figs. 5, 18, 20: C1/470/570; ¶¶0055, 0071, 0091, 0099 – FIG. 20 shows an unoccluded image/entity controller pose to the left of an occluded image/part pose; Pekkucuksen: ¶¶0098, 0101; Wu: ¶0024).
Other Relevant Prior Art
10. Other relevant prior art includes:
U.S. Patent Pub. No. 2016/0267634 A1 to Nam et al. discloses capturing first and second partial images using different filter effects and using then merging the two images into a third image (Abstract; ¶¶0047-0052).
Conclusion
11. THIS ACTION IS MADE NON-FINAL.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIRK W HERMANN whose telephone number is (571) 270-3891. The examiner can normally be reached on Monday-Friday, 10am-7pm, EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, LunYi Lao can be reached on (571) 272-7671. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. If you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KIRK W HERMANN/Primary Examiner, Art Unit 2619
Prosecution Insights