Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claims 30, 45, 48 and 49 are objected to because of the following informalities:
For claim 30, Examiner believes this claim should be amended in the following manner:
A method comprising:
obtaining at least one digital image of a three-dimensional (3D) environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
For claim 45, Examiner believes this claim should be amended in the following manner:
The method of claim 30, wherein the augmented reality content includes two-dimensional (2D) or 3D game content.
For claim 48, Examiner believes this claim should be amended in the following manner:
A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a three-dimensional (3D) environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
For claim 49, Examiner believes this claim should be amended in the following manner:
A system comprising:
a digital camera operable to obtain at least one digital image of a three-dimensional (3D) environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image, identify at least one time varying content anchor associated with the one or more identified objects, obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 30-49 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
For independent claim 30, this claim establishes “augmented reality content” and “obtained augmented reality content”. Claim 30 goes on to recite the phrase “the augmented reality content” and it is unclear and ambiguous to which of the previously established “augmented reality content” and “obtained augmented reality content” is being referenced by the phrase “the augmented reality content”. Examiner has suggested amendments in the claim objections discussed above to resolve the ambiguities.
For dependent claims 31-47, these claims depend from claim 30 and accordingly inherit the deficiencies of claim 30. Therefore, claims 31-47 are likewise indefinite.
For independent claim 48, this claim establishes “augmented reality content” and “obtained augmented reality content”. Claim 48 goes on to recite the phrase “the augmented reality content” and it is unclear and ambiguous to which of the previously established “augmented reality content” and “obtained augmented reality content” is being referenced by the phrase “the augmented reality content”. Examiner has suggested amendments in the claim objections discussed above to resolve the ambiguities.
For independent claim 49, this claim establishes “augmented reality content” and “obtained augmented reality content”. Claim 49 goes on to recite the phrase “the augmented reality content” and it is unclear and ambiguous to which of the previously established “augmented reality content” and “obtained augmented reality content” is being referenced by the phrase “the augmented reality content”. Examiner has suggested amendments in the claim objections discussed above to resolve the ambiguities.
Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 30-33, 36-38, 42, 43 and 45-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 8-11, 13, 23 and 24 of U.S. Patent No. 10,008,047 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
The following is a claim comparison of claims 30-33, 36-38, 42, 43 and 45-49 of the instant application and claims 1, 3, 8-11, 13, 23 and 24 of U.S. Patent No. 10,008,047.
Application No. 19/009,671
U.S. Patent No. 10,008,047
30. A method comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. A chroma key content management system comprising: a chroma key content source interface configured to provide access to a content source storing addressable chroma key content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the chroma key content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
derive environment attributes from the digital representation;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the 3D environment based on a contextual trigger as a function of the one or more recognized objects;
obtain chroma key content via the chroma key content source interface, at least in part, based on the environment attributes;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the chroma key content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the chroma key content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
3. The system of claim 1, wherein the digital representation comprises at least one of the following data modalities: image data, audio data, tactile data, kinesthetic data, gesture data, location data, time data, weather data, biometric data, product data, position data, and user data.
13. The system of claim 1, wherein the at least one content anchor comprises a time-varying content anchor.
31
14
32
14
33
14
36
14
37
14
38
10 and 11
42
1
43
23
45
24
46
9
47
8
48. A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. A chroma key content management system comprising: a chroma key content source interface configured to provide access to a content source storing addressable chroma key content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the chroma key content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
derive environment attributes from the digital representation;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the 3D environment based on a contextual trigger as a function of the one or more recognized objects;
obtain chroma key content via the chroma key content source interface, at least in part, based on the environment attributes;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the chroma key content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the chroma key content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
3. The system of claim 1, wherein the digital representation comprises at least one of the following data modalities: image data, audio data, tactile data, kinesthetic data, gesture data, location data, time data, weather data, biometric data, product data, position data, and user data.
13. The system of claim 1, wherein the at least one content anchor comprises a time-varying content anchor.
49. A system comprising: a digital camera operable to
obtain at least one digital image of a 3D environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image,
identify at least one time varying content anchor associated with the one or more identified objects,
obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and
establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; and
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. A chroma key content management system comprising: a chroma key content source interface configured to provide access to a content source storing addressable chroma key content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the chroma key content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
derive environment attributes from the digital representation;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the 3D environment based on a contextual trigger as a function of the one or more recognized objects;
obtain chroma key content via the chroma key content source interface, at least in part, based on the environment attributes;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the chroma key content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the chroma key content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
3. The system of claim 1, wherein the digital representation comprises at least one of the following data modalities: image data, audio data, tactile data, kinesthetic data, gesture data, location data, time data, weather data, biometric data, product data, position data, and user data.
13. The system of claim 1, wherein the at least one content anchor comprises a time-varying content anchor.
Claims 30-33, 36-38, 42, 43 and 45-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 8-11, 13, 23 and 24 of U.S. Patent No. 10,008,047 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
For claims 30, 48 and 49, claims 1, 3 and 13 of U.S. Patent No. 10,008,047 does not specifically disclose augmented reality content, rendering the augmented reality content with respect to a point of view of a digital camera and determining the augmented reality content differently at different times. However, these limitations are well-known in the art as disclosed in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). It would have been obvious to a person having ordinary skill in the art to apply the use of chroma key content in augmented reality applications to provide virtual scenes to facilitate training and maintenance scenarios (page 1/Section 1) and to render the augmented reality content with respect to a point of view of a user’s head position for mounting a head mounted display device and digital camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12) as taught in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003. It would have been obvious to a person having ordinary skill in the art to apply the use cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107) as taught in Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). Claims 1, 3 and 13 of U.S. Patent No. 10,008,047 otherwise recites identical limitations as shown in the claim chart above. Thus, claims 30, 48 and 49 of the instant application are not patentably distinct from claims 1, 3 and 13 of U.S. Patent No. 10,008,047.
Similarly, for claims 31-33, 36-38, 42, 43 and 45-47, claims 1, 8-11, 13, 23 and 24 of U.S. Patent No. 10,008,047 mirror the limitations of claims 31-33, 36-38, 42, 43 and 45-47 as set forth in the claim chart above. Thus, claims 31-33, 36-38, 42, 43 and 45-47 of the instant application are not patentably distinct from claims , 8-11, 13, 23 and 24 of U.S. Patent No. 10,008,047.
Claims 30, 38, 42, 43 and 46-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 11 and 21-24 of U.S. Patent No. 10,019,847 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
The following is a claim comparison of claims 30, 38, 42, 43 and 46-49 of the instant application and claims 1, 2, 11 and 21-24 of U.S. Patent No. 10,019,847.
Application No. 19/009,671
U.S. Patent No. 10,019,847
30. A method comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. A chroma key content management system comprising: a chroma key content source interface configured to provide access to a content source storing addressable chroma key content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the chroma key content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment; derive environment attributes from the digital representation;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with a human body based on a contextual trigger as a function of the one or more recognized objects;
obtain chroma key content via the chroma key content source interface, at least in part, based on the environment attributes, the chroma key content including a representation of medical information;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the chroma key content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the chroma key content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
11. The system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
38
23 and 24
42
1
43
2
46
22
47
21
48. A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. A chroma key content management system comprising: a chroma key content source interface configured to provide access to a content source storing addressable chroma key content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the chroma key content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment; derive environment attributes from the digital representation;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with a human body based on a contextual trigger as a function of the one or more recognized objects;
obtain chroma key content via the chroma key content source interface, at least in part, based on the environment attributes, the chroma key content including a representation of medical information;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the chroma key content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the chroma key content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
11. The system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
49. A system comprising: a digital camera operable to
obtain at least one digital image of a 3D environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image,
identify at least one time varying content anchor associated with the one or more identified objects,
obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and
establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; and
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. A chroma key content management system comprising: a chroma key content source interface configured to provide access to a content source storing addressable chroma key content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the chroma key content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment; derive environment attributes from the digital representation;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with a human body based on a contextual trigger as a function of the one or more recognized objects;
obtain chroma key content via the chroma key content source interface, at least in part, based on the environment attributes, the chroma key content including a representation of medical information;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the chroma key content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the chroma key content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
11. The system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
Claims 30, 38, 42, 43 and 46-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 11 and 21-24 of U.S. Patent No. 10,019,847 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
For claims 30, 48 and 49, claims 1 and 11 of U.S. Patent No. 10,019,847 does not specifically disclose augmented reality content, rendering the augmented reality content for a time varying content anchor with respect to a point of view of a digital camera and determining the augmented reality content differently at different times. However, these limitations are well-known in the art as disclosed in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). It would have been obvious to a person having ordinary skill in the art to apply the use of chroma key content in augmented reality applications to provide virtual scenes to facilitate training and maintenance scenarios (page 1/Section 1) and to render the augmented reality content for a time varying content anchor with respect to a point of view of a user’s head position for mounting a head mounted display device and digital camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12) as taught in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003. It would have been obvious to a person having ordinary skill in the art to apply the use cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107) as taught in Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). Claims 1 and 11 of U.S. Patent No. 10,019,847 otherwise recites identical limitations as shown in the claim chart above. Thus, claims 30, 48 and 49 of the instant application are not patentably distinct from claims 1 and 11 of U.S. Patent No. 10,019,847.
Similarly, for claims 38, 42, 43, 46 and 47, claims 1, 2 and 21-24 of U.S. Patent No. 10,019,847 mirror the limitations of claims 38, 42, 43, 46 and 47 as set forth in the claim chart above. Thus, claims 38, 42, 43, 46 and 47 of the instant application are not patentably distinct from claims 1, 2 and 21-24 of U.S. Patent No. 10,019,847.
Claims 30, 42-44 and 46-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4, 5, 7, 8, 12 and 13 of U.S. Patent No. 10,255,730 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
The following is a claim comparison of claims 30, 42-44 and 46-49 of the instant application and claims 1, 4, 5, 7, 8, 12 and 13 of U.S. Patent No. 10,255,730.
Application No. 19/009,671
U.S. Patent No. 10,255,730
30. A method comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising: a content source interface configured to provide access to a content source storing addressable augmented reality content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the 3D environment as a function of the one or more recognized objects;
obtain augmented reality content via the content source interface;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the augmented reality content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the augmented reality content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
8. The system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
42
4
43
5
44
7
46
12
47
13
48. A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising: a content source interface configured to provide access to a content source storing addressable augmented reality content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the 3D environment as a function of the one or more recognized objects;
obtain augmented reality content via the content source interface;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the augmented reality content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the augmented reality content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
8. The system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
49. A system comprising: a digital camera operable to
obtain at least one digital image of a 3D environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image,
identify at least one time varying content anchor associated with the one or more identified objects,
obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and
establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; and
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising: a content source interface configured to provide access to a content source storing addressable augmented reality content;
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine coupled with the content source interface and executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
execute one or more object recognition algorithms on the digital representation to recognize one or more objects from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the 3D environment as a function of the one or more recognized objects;
obtain augmented reality content via the content source interface;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the augmented reality content and the at least one content anchor; and
configure an output device to render, at least in part, an environmental model of the 3D environment based on the digital representation and render the augmented reality content at the content position and content orientation within the environmental model with respect to a point of view from which the environmental model is rendered.
8. The system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
Claims 30, 42-44 and 46-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4, 5, 7, 8, 12 and 13 of U.S. Patent No. 10,255,730 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
For claims 30, 48 and 49, claims 1 and 8 of U.S. Patent No. 10,255,730 does not specifically disclose augmented reality content, rendering the augmented reality content for a time varying content anchor with respect to a point of view of a digital camera and determining the augmented reality content differently at different times. However, these limitations are well-known in the art as disclosed in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). It would have been obvious to a person having ordinary skill in the art to apply the use of chroma key content in augmented reality applications to provide virtual scenes to facilitate training and maintenance scenarios (page 1/Section 1) and to render the augmented reality content for a time varying content anchor with respect to a point of view of a user’s head position for mounting a head mounted display device and digital camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12) as taught in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003. It would have been obvious to a person having ordinary skill in the art to apply the use cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107) as taught in Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). Claims 1 and 8 of U.S. Patent No. 10,255,730 otherwise recites identical limitations as shown in the claim chart above. Thus, claims 30, 48 and 49 of the instant application are not patentably distinct from claims 1 and 8 of U.S. Patent No. 10,255,730.
Similarly, for claims 42-44 and 46-47, claims 4, 5, 7, 12 and 13 of U.S. Patent No. 10,255,730 mirror the limitations of claims 42-44 and 46-47as set forth in the claim chart above. Thus, claims 42-44 and 46-47 of the instant application are not patentably distinct from claims 4, 5, 7, 12 and 13 of U.S. Patent No. 10,255,730.
Claims 30, 42-44 and 46-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4, 5, 7, 8, 12 and 13 of U.S. Patent No. 10,733,808 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
The following is a claim comparison of claims 30, 42-44 and 46-49 of the instant application and claims 1, 4, 5, 7, 8, 12 and 13 of U.S. Patent No. 10,733,808.
Application No. 19/009,671
U.S. Patent No. 10,733,808
30. A method comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
execute one or more object recognition algorithms on the digital representation to identify one or more surfaces from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the one or more identified surfaces;
obtain augmented reality content from a content source;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the augmented reality content and the at least one content anchor; and
configure an output device to render the 3D environment and to render the augmented reality content at the content position and content orientation within an environmental model of the 3D environment with respect to a point of view of a camera, the environmental model being based on the digital representation and representing the one or more identified surfaces.
8. The augmented reality system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
42
4
43
5
44
7
46
12
47
13
48. A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
execute one or more object recognition algorithms on the digital representation to identify one or more surfaces from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the one or more identified surfaces;
obtain augmented reality content from a content source;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the augmented reality content and the at least one content anchor; and
configure an output device to render the 3D environment and to render the augmented reality content at the content position and content orientation within an environmental model of the 3D environment with respect to a point of view of a camera, the environmental model being based on the digital representation and representing the one or more identified surfaces.
8. The augmented reality system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
49. A system comprising: a digital camera operable to
obtain at least one digital image of a 3D environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image,
identify at least one time varying content anchor associated with the one or more identified objects,
obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and
establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; and
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain a digital representation of a 3D environment;
execute one or more object recognition algorithms on the digital representation to identify one or more surfaces from among a plurality of objects in the digital representation;
identify at least one content anchor associated with the one or more identified surfaces;
obtain augmented reality content from a content source;
establish a content position and content orientation relative to the at least one content anchor as a function of content attributes of the augmented reality content and the at least one content anchor; and
configure an output device to render the 3D environment and to render the augmented reality content at the content position and content orientation within an environmental model of the 3D environment with respect to a point of view of a camera, the environmental model being based on the digital representation and representing the one or more identified surfaces.
8. The augmented reality system of claim 1, wherein the digital representation includes a digital image and the content management engine is configured to obtain the digital image via a web camera.
Claims 30, 42-44 and 46-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4, 5, 7, 8, 12 and 13 of U.S. Patent No. 10,733,808 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
For claims 30, 48 and 49, claims 1 and 8 of U.S. Patent No. 10,733,808 does not specifically disclose a time varying content anchor and determining augmented reality content differently at different times. However, these limitations are well-known in the art as disclosed in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). It would have been obvious to a person having ordinary skill in the art to apply the use of chroma key content in augmented reality applications to provide virtual scenes to facilitate training and maintenance scenarios (page 1/Section 1) and to render the augmented reality content for a time varying content anchor with respect to a point of view of a user’s head position for mounting a head mounted display device and digital camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12) as taught in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003. It would have been obvious to a person having ordinary skill in the art to apply the use cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107) as taught in Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). Claims 1 and 8 of U.S. Patent No. 10,733,808 otherwise recites identical limitations as shown in the claim chart above. Thus, claims 30, 48 and 49 of the instant application are not patentably distinct from claims 1 and 8 of U.S. Patent No. 10,733,808.
Similarly, for claims 42-44 and 46-47, claims 4, 5, 7, 12 and 13 of U.S. Patent No. 10,733,808 mirror the limitations of claims 42-44 and 46-47 as set forth in the claim chart above. Thus, claims 42-44 and 46-47 of the instant application are not patentably distinct from claims 4, 5, 7, 12 and 13 of U.S. Patent No. 10,733,808.
Claims 30, 38, 39 and 42-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 5, 6, 9, 10, 13, 16 and 17 of U.S. Patent No. 11,495,001 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
The following is a claim comparison of claims 30, 38, 39 and 42-49 of the instant application and claims 1, 2, 5, 6, 9, 10, 13, 16 and 17 of U.S. Patent No. 11,495,001.
Application No. 19/009,671
U.S. Patent No. 11,495,001
30. A method comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects; obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a real-world room;
execute one or more object recognition algorithms on the at least one digital image to identify one or more surfaces from among a plurality of objects in the at least one digital image;
obtain augmented reality content from a content source, the augmented reality content having one or more characteristics; select at least one surface from among the one or more identified surfaces based on a comparison between a surface area of the at least one surface with the one or more characteristics of the augmented reality content, the at least one surface being selected only if the surface area is sufficiently large enough for the augmented reality content; identify at least one content anchor associated with the selected at least one surface;
establish a content position and a content orientation relative to the at least one content anchor; and
configure an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the real-world room, at least a portion of the displayed image of the real-world room extending beyond the at least one content anchor, wherein the augmented reality content is subsequently movable around on the selected at least one surface within the displayed image of the real-world room in response to an instruction received via an input device.
38
2
39
5
42
6
43
9
44
10
45
13
46
16
47
17
48. A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects; obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a real-world room;
execute one or more object recognition algorithms on the at least one digital image to identify one or more surfaces from among a plurality of objects in the at least one digital image;
obtain augmented reality content from a content source, the augmented reality content having one or more characteristics; select at least one surface from among the one or more identified surfaces based on a comparison between a surface area of the at least one surface with the one or more characteristics of the augmented reality content, the at least one surface being selected only if the surface area is sufficiently large enough for the augmented reality content; identify at least one content anchor associated with the selected at least one surface;
establish a content position and a content orientation relative to the at least one content anchor; and
configure an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the real-world room, at least a portion of the displayed image of the real-world room extending beyond the at least one content anchor, wherein the augmented reality content is subsequently movable around on the selected at least one surface within the displayed image of the real-world room in response to an instruction received via an input device.
49. A system comprising: a digital camera operable to
obtain at least one digital image of a 3D environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image,
identify at least one time varying content anchor associated with the one or more identified objects, obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; and
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a real-world room;
execute one or more object recognition algorithms on the at least one digital image to identify one or more surfaces from among a plurality of objects in the at least one digital image;
obtain augmented reality content from a content source, the augmented reality content having one or more characteristics; select at least one surface from among the one or more identified surfaces based on a comparison between a surface area of the at least one surface with the one or more characteristics of the augmented reality content, the at least one surface being selected only if the surface area is sufficiently large enough for the augmented reality content; identify at least one content anchor associated with the selected at least one surface;
establish a content position and a content orientation relative to the at least one content anchor; and
configure an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the real-world room, at least a portion of the displayed image of the real-world room extending beyond the at least one content anchor, wherein the augmented reality content is subsequently movable around on the selected at least one surface within the displayed image of the real-world room in response to an instruction received via an input device.
Claims 30, 38, 39 and 42-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 5, 6, 9, 10, 13, 16 and 17 of U.S. Patent No. 11,495,001 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
For claims 30, 48 and 49, claim 1 of U.S. Patent No. 11,495,001 does not specifically disclose a time varying content anchor and determining augmented reality content differently at different times. However, these limitations are well-known in the art as disclosed in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). It would have been obvious to a person having ordinary skill in the art to apply the use of chroma key content in augmented reality applications to provide virtual scenes to facilitate training and maintenance scenarios (page 1/Section 1) and to render the augmented reality content for a time varying content anchor with respect to a point of view of a user’s head position for mounting a head mounted display device and digital camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12) as taught in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003. It would have been obvious to a person having ordinary skill in the art to apply the use cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107) as taught in Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). Claim 1 of U.S. Patent No. 11,495,001 otherwise recites identical limitations as shown in the claim chart above. Thus, claims 30, 48 and 49 of the instant application are not patentably distinct from claim 1 of U.S. Patent No. 11,495,001.
Similarly, for claims 38, 39 and 42-47, claims 2, 5, 6, 9, 10, 13, 16 and 17 of U.S. Patent No. 11,495,001 mirror the limitations of claims 38, 39 and 42-47 as set forth in the claim chart above. Thus, claims 38, 39 and 42-47 of the instant application are not patentably distinct from claims 2, 5, 6, 9, 10, 13, 16 and 17 of U.S. Patent No. 11,495,001.
Claims 30 and 38-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 5-8, 11, 12, 15, 17 and 18 of U.S. Patent No. 12,223,610 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
The following is a claim comparison of claims 30 and 38-49 of the instant application and claims 1, 2, 5-8, 11, 12, 15, 17 and 18 of U.S. Patent No. 12,223,610.
Application No. 19/009,671
U.S. Patent No. 12,223,610
30. A method comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a 3D environment;
execute one or more object recognition algorithms on the at least one digital image to identify one or more surfaces from among a plurality of objects in the at least one digital image;
identify at least one content anchor associated with the one or more identified surfaces;
obtain first augmented reality content from a content source;
establish a first content position and a first content orientation relative to the at least one content anchor;
configure an output device to render and display, with respect to a point of view of a camera, the first augmented reality content at the first content position and the first content orientation within a displayed image of the 3D environment, at least a portion of the displayed image of the 3D environment extending beyond the at least one content anchor;
obtain second augmented reality content from the content source;
establish a second content position and a second content orientation relative to the first augmented reality content; and configure the output device to render and display the second augmented reality content at the second content position and the second content orientation within the displayed image of the 3D environment, wherein the content management engine configures the output device to render the second augmented reality content as a digital representation of a commercial product overlaid on the first augmented reality content.
38
2
39
5
40
6
41
7
42
8
43
11
44
12
45
15
46
17
47
18
48. A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a 3D environment;
execute one or more object recognition algorithms on the at least one digital image to identify one or more surfaces from among a plurality of objects in the at least one digital image;
identify at least one content anchor associated with the one or more identified surfaces;
obtain first augmented reality content from a content source;
establish a first content position and a first content orientation relative to the at least one content anchor;
configure an output device to render and display, with respect to a point of view of a camera, the first augmented reality content at the first content position and the first content orientation within a displayed image of the 3D environment, at least a portion of the displayed image of the 3D environment extending beyond the at least one content anchor;
obtain second augmented reality content from the content source;
establish a second content position and a second content orientation relative to the first augmented reality content; and configure the output device to render and display the second augmented reality content at the second content position and the second content orientation within the displayed image of the 3D environment, wherein the content management engine configures the output device to render the second augmented reality content as a digital representation of a commercial product overlaid on the first augmented reality content.
49. A system comprising: a digital camera operable to
obtain at least one digital image of a 3D environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image,
identify at least one time varying content anchor associated with the one or more identified objects,
obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and
establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; and
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and
a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a 3D environment;
execute one or more object recognition algorithms on the at least one digital image to identify one or more surfaces from among a plurality of objects in the at least one digital image;
identify at least one content anchor associated with the one or more identified surfaces;
obtain first augmented reality content from a content source;
establish a first content position and a first content orientation relative to the at least one content anchor;
configure an output device to render and display, with respect to a point of view of a camera, the first augmented reality content at the first content position and the first content orientation within a displayed image of the 3D environment, at least a portion of the displayed image of the 3D environment extending beyond the at least one content anchor;
obtain second augmented reality content from the content source;
establish a second content position and a second content orientation relative to the first augmented reality content; and configure the output device to render and display the second augmented reality content at the second content position and the second content orientation within the displayed image of the 3D environment, wherein the content management engine configures the output device to render the second augmented reality content as a digital representation of a commercial product overlaid on the first augmented reality content.
Claims 30 and 38-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 5-8, 11, 12, 15, 17 and 18 of U.S. Patent No. 12,223,610 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
For claims 30, 48 and 49, claim 1 of U.S. Patent No. 12,223,610 does not specifically disclose a time varying content anchor and determining augmented reality content differently at different times. However, these limitations are well-known in the art as disclosed in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). It would have been obvious to a person having ordinary skill in the art to apply the use of chroma key content in augmented reality applications to provide virtual scenes to facilitate training and maintenance scenarios (page 1/Section 1) and to render the augmented reality content for a time varying content anchor with respect to a point of view of a user’s head position for mounting a head mounted display device and digital camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12) as taught in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003. It would have been obvious to a person having ordinary skill in the art to apply the use cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107) as taught in Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). Claim 1 of U.S. Patent No. 12,223,610 otherwise recites identical limitations as shown in the claim chart above. Thus, claims 30, 48 and 49 of the instant application are not patentably distinct from claim 1 of U.S. Patent No. 12,223,610.
Similarly, for claims 38-47, claims 2, 5-8, 11, 12, 15, 17 and 18 of U.S. Patent No. 12,223,610 mirror the limitations of claims 38-47 as set forth in the claim chart above. Thus, claims 38-47 of the instant application are not patentably distinct from claims 2, 5-8, 11, 12, 15, 17 and 18 of U.S. Patent No. 12,223,610.
Claims 30, 36 and 38-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 4-9 and 11-13 of U.S. Patent No. 12,229,910 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
The following is a claim comparison of claims 30, 36 and 38-49 of the instant application and claims 1, 2, 4-9 and 11-13 of U.S. Patent No. 12,229,910.
Application No. 19/009,671
U.S. Patent No. 12,229,910
30. A method comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising:
at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a 3D environment;
execute one or more recognition algorithms on the at least one digital image to identify one or more surfaces in the at least one digital image;
identify at least one content anchor associated with the one or more identified surfaces; obtain location data of the 3D environment;
obtain augmented reality content from a content source as a function of the location data;
establish a content position and a content orientation relative to the at least one content anchor;
configure an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment, at least a portion of the displayed image of the 3D environment extending beyond the at least one content anchor, wherein said establishing the content position and the content orientation relative to the at least one content anchor includes establishing the content position and the content orientation relative to a virtual structure that is rendered relative to the at least one content anchor; and place advertisements over the virtual structure with respect to the point of view of the camera.
36
1
38
2
39
4
40
5
41
6
42
7
43
8
44
9
45
11
46
12
47
13
48. A computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors to perform operations comprising:
obtaining at least one digital image of a 3D environment;
executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image;
identifying at least one time varying content anchor associated with the one or more identified objects;
obtaining augmented reality content from a content source as a function of the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times;
establishing a content position and a content orientation relative to the at least one time varying content anchor; and
configuring an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising: at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a 3D environment;
execute one or more recognition algorithms on the at least one digital image to identify one or more surfaces in the at least one digital image;
identify at least one content anchor associated with the one or more identified surfaces; obtain location data of the 3D environment;
obtain augmented reality content from a content source as a function of the location data;
establish a content position and a content orientation relative to the at least one content anchor;
configure an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment, at least a portion of the displayed image of the 3D environment extending beyond the at least one content anchor, wherein said establishing the content position and the content orientation relative to the at least one content anchor includes establishing the content position and the content orientation relative to a virtual structure that is rendered relative to the at least one content anchor; and place advertisements over the virtual structure with respect to the point of view of the camera.
49. A system comprising: a digital camera operable to
obtain at least one digital image of a 3D environment;
one or more processors configured to execute one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image,
identify at least one time varying content anchor associated with the one or more identified objects,
obtain augmented reality content from a content source as a function of the at least one time varying content anchor, and
establish a content position and a content orientation relative to the at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times; and
an output device configured to render and display, with respect to a point of view of the camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment.
1. An augmented reality system comprising: at least one non-transitory computer readable medium storing software instructions executable by at least one processor; and a content management engine executable on the at least one processor according to the stored software instructions to:
obtain at least one digital image of a 3D environment;
execute one or more recognition algorithms on the at least one digital image to identify one or more surfaces in the at least one digital image;
identify at least one content anchor associated with the one or more identified surfaces; obtain location data of the 3D environment;
obtain augmented reality content from a content source as a function of the location data;
establish a content position and a content orientation relative to the at least one content anchor;
configure an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position and the content orientation within a displayed image of the 3D environment, at least a portion of the displayed image of the 3D environment extending beyond the at least one content anchor, wherein said establishing the content position and the content orientation relative to the at least one content anchor includes establishing the content position and the content orientation relative to a virtual structure that is rendered relative to the at least one content anchor; and place advertisements over the virtual structure with respect to the point of view of the camera.
Claims 30, 36 and 38-49 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 4-9 and 11-13 of U.S. Patent No. 12,229,910 in view of Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1).
For claims 30, 48 and 49, claim 1 of U.S. Patent No. 12,229,910 does not specifically disclose a time varying content anchor and determining augmented reality content differently at different times. However, these limitations are well-known in the art as disclosed in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003, and Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). It would have been obvious to a person having ordinary skill in the art to apply the use of chroma key content in augmented reality applications to provide virtual scenes to facilitate training and maintenance scenarios (page 1/Section 1) and to render the augmented reality content for a time varying content anchor with respect to a point of view of a user’s head position for mounting a head mounted display device and digital camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12) as taught in Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003. It would have been obvious to a person having ordinary skill in the art to apply the use cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107) as taught in Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1). Claim 1 of U.S. Patent No. 12,229,910 otherwise recites identical limitations as shown in the claim chart above. Thus, claims 30, 48 and 49 of the instant application are not patentably distinct from claim 1 of U.S. Patent No. 12,229,910.
Similarly, for claims 36 and 38-47, claims 2, 4-9 and 11-13 of U.S. Patent No. 12,229,910 mirror the limitations of claims 36 and 38-47 as set forth in the claim chart above. Thus, claims 36 and 38-47 of the instant application are not patentably distinct from claims 2, 4-9 and 11-13 of U.S. Patent No. 12,229,910.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 30, 34, 36, 38-43, 45, 46, 48 and 49 are rejected under 35 U.S.C. 103 as being unpatentable over Nakajima et al., A Support System for Maintenance Training by Augmented Reality, Proceedings of the 12th International Conference on Image Analysis and Processing, September 2003 (hereinafter “Nakajima”) in view of Flaks et al. (U.S. Patent Application Publication 2012/0092328 A1, hereinafter “Flaks”) (all references made of record of the IDS submitted 1/24/2025).
For claim 30, Nakajima discloses a method (disclosing a system and method for managing CG images for chroma-key image composition with real-world images as chroma key content for presenting augmented reality (pages 1-2/Sections 1-2.1/Fig. 2)) comprising: obtaining at least one digital image of a 3D environment (disclosing a camera to capture an image of a real-world environment as a digital representation for processing by a workstation to perform chroma-key image composition of 3D CG images for generating a 3D augmented reality environment (page 2/Section 2.1/Fig. 2, page 3/Section 2.3 and page 4/Section 3.1.2)); executing one or more recognition algorithms on the at least one digital image to identify one or more objects in the at least one digital image (disclosing the workstation executes an Object Detection module as object recognition software to perform object recognition on the captured image page 2/Section 2.1-2.2/Fig. 2)); identify at least one time varying content anchor associated with the one or more identified objects (disclosing the workstation detects and identifies a moving object that moves over time as a time varying content anchor for chroma-key composition with corresponding 3D CG images within the 3D augmented reality environment based on the detection of a blue screen as a contextual trigger within the captured image to facilitate the detection of the object (page 2/Section 2.1/Fig. 2 and page 4/Section 3.1.2)); obtain augmented reality content from a content source (disclosing the workstation obtains 3D CG images from a database based on determined environment features and attributes (page 3/Section 2.3, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10 and 12)); establish a content position relative to the at least one time varying content anchor (disclosing the 3D CG images for chroma-key image composition are positioned relative to the recognized object as the content anchor where differences in the content attributes and types of the recognized object may result in differences in positioning the corresponding 3D CG images (pages 5-6/Sections 3.4-3.4.2) such that a recognized model of a nuclear power plant may simply have a CG text overlaid within the boundaries of the model of the nuclear power plant (page 5/Sections 3.4.1/Figs. 11) and a recognized fire extinguisher nozzle may have CG fire-extinguishing liquid spouted from the nozzle (page 6/Section 3.4.2/Fig. 12)); and configure an output device to render and display, with respect to a point of view of a camera, the augmented reality content at the content position within a displayed image of the 3D environment (disclosing the workstation configures a head-mounted display (HMD) as an output device for performing chroma-key image composition to render the 3D CG images at their appropriate positions relative to the detected object within the captured image of the real world environment with respect to a camera on the HMD within the 3D augmented reality environment at a point of view of a user's head position for mounting the HMD and the camera (page 2/Section 2.1/Fig. 1, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10-12)).
Nakajima does not disclose obtaining augmented reality content as a function of at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times, and establishing a content orientation and rendering and displaying augmented reality content at the content orientation.
However, these limitations are well-known in the art as disclosed in Flaks.
Flaks similarly discloses a system and method for positioning virtual image content as augmented reality content with respect to corresponding real world objects to present augmented reality (page 1/par. 4). Flaks likewise discloses cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects as time varying content anchors (page 9/par. 99) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107). Flaks explains its system performs object identification and recognition on the captured images to identify distinct objects from each other within the captured images such as identifying a person, table and a chair within a room of persons, chairs, table, couches, etc. for overlaying appropriate virtual objects over the identified objects (pages 10-11/par. 102, 103 and 105). Flaks further explains its system performs skeletal tracking for identifying and tracking the different body parts as different objects of the persons identified within the captured images such that virtual objects may be overlaid over a particular body part of the identified persons, e.g. overlaying a new pair of pants over the identified legs of an identified person (page 7/par. 76 and pages 10-11/par. 102 and 105). Flaks further explains its system generates a three dimensional environmental model for a 3D augmented reality environment for rendering the virtual image content at the object position and orientation within the model (page 6/par. 61, page 10/par. 102 and page 11/par. 108-111). It follows Nakajima may be accordingly modified with the teachings of Flaks for obtaining its augmented reality content as a function of its at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times, and establishing a content orientation corresponding to its detected object for rendering its chroma key content at the content position and content orientation for its 3D augmented reality environment.
A person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention would find it obvious to modify Nakajima with the teachings of Flaks. Flaks is analogous art in dealing with a system and method for positioning virtual image content with respect to corresponding real world objects to present augmented reality (page 1/par. 4). Flaks discloses its generation of an environmental model is advantageous in tracking the movement of real world objects such that corresponding virtual content is appropriately positioned for rendering with respect to the real world objects (page 11/par. 108-111). Consequently, a PHOSITA would incorporate the teachings of Flaks into Nakajima for tracking the movement of real world objects such that corresponding virtual content is appropriately positioned for rendering with respect to the real world objects.
For claim 34, depending on claim 30, Nakajima as modified by Flaks discloses wherein the at least one time varying content anchor changes according to motion of the one or more identified objects (Flaks similarly discloses a system and method for positioning virtual image content as augmented reality content with respect to corresponding real world objects to present augmented reality (page 1/par. 4); Flaks likewise discloses cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location, orientation and motion of moving real world objects as time varying content anchors (page 9/par. 99; and page 11/par. 112) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107); and it follows Nakajima may be accordingly modified with the teachings of Flaks for obtaining its augmented reality content as a function of its at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times).
For claim 36, depending on claim 30, Nakajima as modified by Flaks discloses wherein the at least one time varying content anchor comprises a location (Flaks similarly discloses a system and method for positioning virtual image content as augmented reality content with respect to corresponding real world objects to present augmented reality (page 1/par. 4); Flaks likewise discloses cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location, orientation and motion of moving real world objects as time varying content anchors (page 9/par. 99; and page 11/par. 112) such that virtual image content are obtained from a content source as a function of the moving real world objects and determined differently at different times through adjustment to match the appropriate orientation, size and shape of the corresponding real world objects (pages 5-6/par. 59 and 61; and page 11/par. 107); and it follows Nakajima may be accordingly modified with the teachings of Flaks for obtaining its augmented reality content as a function of its at least one time varying content anchor, wherein the obtained augmented reality content is determined differently at different times).
For claim 38, depending on claim 30, Nakajima as modified by Flaks discloses wherein the at least one time varying content anchor is derived from one or more feature descriptors of the at least one digital image (Flaks discloses the use of profiles with data for describing features (descriptors) to be detected and matched within captured images for determining the real world objects within the environment (pages 10-11/par. 102 and 105-106) and it follows Nakajima may be accordingly modified with the teachings of Flaks for incorporating the use of profiles with data for describing features to be detected for detecting objects as its time varying content anchors within its captured images).
For claim 39, depending on claim 30, Nakajima as modified by Flaks discloses wherein said identifying at least one time varying content anchor comprises receiving a user's selection of the at least one time varying content anchor (Flaks discloses its augmented reality system allows a user to select and indicate a particular real world object as a content anchor to be entirely or partially replaced with a corresponding virtual object (page 10/par. 101-102); and it follows Nakajima may be accordingly modified with the teachings of Flaks implement user selection of its time varying content anchor).
For claim 40, depending on claim 30, Nakajima as modified by Flaks discloses wherein said identifying the at least one time varying content anchor comprises determining the at least one time varying content anchor based on one or more characteristics of a plurality of objects in the at least one digital image (Nakajima discloses the workstation detects and recognizes objects within the captured image (page 2/Section 2.1/Fig. 2 and page 4/Section 3.1.2) to determine characteristics such as recognizing tools for use in an electric power plant environment or recognizing a fire extinguisher nozzle for use in a fire-fighting environment (pages 5-6/Sections 3.4-3.4.2/Figs. 10 and 12); Nakajima discloses a recognized model of a nuclear power plant is determined as a content anchor to have a CG text overlaid within the boundaries of the model of the nuclear power plant (page 5/Sections 3.4.1/Figs. 11) and a recognized fire extinguisher nozzle is likewise determined as a time varying content anchor to have CG fire-extinguishing liquid spouted from the nozzle (page 6/Section 3.4.2/Fig. 12)).
For claim 41, depending on claim 30, Nakajima as modified by Flaks discloses wherein said identifying the at least one time varying content anchor comprises determining the at least time varying one content anchor based on one or more characteristics of the augmented reality content (Nakajima discloses the workstation detects and recognizes objects within the captured image (page 2/Section 2.1/Fig. 2 and page 4/Section 3.1.2) to determine characteristics such as recognizing tools for use in an electric power plant environment or recognizing a fire extinguisher nozzle for use in a fire-fighting environment for determining appropriate characteristics and context of the augmented reality content (pages 5-6/Sections 3.4-3.4.2/Figs. 10 and 12); Nakajima discloses a recognized model of a nuclear power plant is determined as a content anchor to have a CG text overlaid within the boundaries of the model of the nuclear power plant (page 5/Sections 3.4.1/Figs. 11) and a recognized fire extinguisher nozzle is likewise determined as a time varying content anchor to have CG fire-extinguishing liquid spouted from the nozzle (page 6/Section 3.4.2/Fig. 12)).
For claim 42, depending on claim 30, Nakajima as modified by Flaks discloses wherein said identifying the at least one time varying content anchor comprises determining the at least one time varying content anchor based on a contextual trigger (Nakajima discloses the workstation detects and identifies an object as a time varying content anchor for chroma-key composition with corresponding 3D CG images within the 3D augmented reality environment based on the detection of a blue screen as a contextual trigger within the captured image to facilitate the detection of the object (page 2/Section 2.1/Fig. 2 and page 4/Section 3.1.2)).
For claim 43, depending on claim 30, Nakajima as modified by Flaks discloses wherein the content source is a database (Nakajima discloses the workstation obtains 3D CG images from a database based on determined environment features and attributes (page 3/Section 2.3, page 4/Section 3.1.2 and pages 5-6/Sections 3.4-3.4.2/Figs. 10 and 12)).
For claim 45, depending on claim 30, Nakajima as modified by Flaks discloses wherein the augmented reality content includes 2D or 3D game content (Nakajima discloses its augmented reality content for chroma-key composition may be 2D or 3D CG (page 3/Section 2.3); Flaks similarly discloses a system and method for positioning virtual image content with respect to corresponding real world objects to present augmented reality (page 1/par. 4); Flaks likewise discloses cameras as capture devices for acquiring digital images of the real world environment (page 2/par. 34-35 and page 4/par. 47) to track the location and orientation of moving real world objects (page 9/par. 99) such that virtual image content are adjusted to match the appropriate orientation of the corresponding real world objects (page 6/par. 61) where the virtual image content may be inserted as part of a video game (page 10/par. 101); and it follows Nakajima may be accordingly modified with the teachings of Flaks for rendering its 2D or 3D CG as 2D or 3D video game content).
For claim 46, depending on claim 30, Nakajima as modified by Flaks discloses wherein the at least one time varying content anchor comprises a vector within an environmental model of the 3D environment (Nakajima discloses a support vector machine for recognizing objects as time varying content anchors according to their corresponding vectors in the captured images (page 2/par. Section 2.2); Flaks further explains it is known to generate a three dimensional environmental model for detecting objects within a 3D augmented reality environment (page 2/par. 29 and page 10/par. 102) and it follows Nakajima may be accordingly modified with the teachings of Flaks for determining vectors to detect objects within an environmental model of its 3D augmented reality environment).
For claim 48, Nakajima as modified by Flaks discloses a computer program product comprising one or more non-transitory computer readable media storing software instructions executable by one or more processors (Flaks similarly discloses a system and method for positioning virtual image content with respect to corresponding real world objects to present augmented reality (page 1/par. 4); Flaks explains its computing system may be implemented with a processor readable storage device for storing software instructions to be executed by a processor (page 2/par. 33) and it follows Nakajima may be accordingly modified with the teachings of Flaks to implement its system of its augmented reality content source interface and its content management engine as software instructions stored on a storage device for execution by a processor) to perform operations comprising the method of claim 30 (see above as to claim 30).
For claim 49, Nakajima as modified by Flaks discloses a system (Nakajima discloses a system includes a computing device as an SGI-Octane workstation for accessing the CG images to perform chroma-key image composition (page 2/Section 2.1/Fig. 2 and page 4/Section 3.1.2.)) comprising: a digital camera (Nakajima discloses a camera as a digital camera (page 2/Section 2.1/Fig. 2, page 3/Section 2.3 and page 4/Section 3.1.2)); one or more processors (Nakajima discloses the workstation includes dual R10000/250Mhz processors (page 2/Section 2.1/Fig. 2 and page 4/Section 3.1.2.)); and an output device (Nakajima discloses a head-mounted display (HMD) as an output device (page 2/Section 2.1/Fig. 1)) to perform the method of claim 30 (see above as to claim 30).
Claim 31 is rejected under 35 U.S.C. 103 as being unpatentable over Nakajima in view of Flaks further in view of Han et al. (U.S. Patent Application Publication 2012/0191737 A1, hereinafter “Han”).
For claim 31, depending on claim 30, Nakajima as modified by Flaks does not disclose an absolute time.
However, these limitations are well-known in the art as disclosed in Han.
Han similarly discloses a system and method for facilitating in interaction between a real world and a virtual world to present augmented reality (page 1/par. 4). Han discloses its system senses motion of objects in a real world with respect to an absolute time (page 2/par. 34 and pages 23-24/par. 480/Table 1-1). It follows Nakajima and Flaks may be accordingly modified with the teachings of Han to change its at least one time varying content anchor according to an absolute time.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Nakajima and Flaks with the teachings of Han. Han is analogous art in dealing with a system and method for facilitating in interaction between a real world and a virtual world to present augmented reality (page 1/par. 4). Han discloses its use of absolute time is advantageous in appropriately sensing and tracking motion of objects in a real world (page 2/par. 34 and pages 23-24/par. 480/Table 1-1). Consequently, a PHOSITA would incorporate the teachings of Han into Nakajima and Flaks for appropriately sensing and tracking motion of objects in a real world. Therefore, claim 31 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
Claim 32 is rejected under 35 U.S.C. 103 as being unpatentable over Nakajima in view of Flaks further in view of Lee et al. (U.S. Patent Application Publication 2013/0249900 A1, hereinafter “Lee”).
For claim 32, depending on claim 30, Nakajima as modified by Flaks does not disclose a relative time.
However, these limitations are well-known in the art as disclosed in Lee.
Lee similarly discloses a system and method for synthesizing information of a real world and a virtual world to present augmented reality (page 1/par. 5). Lee discloses its system determines motion of real world objects with respect to a relative time (pages 3-4/par. 42 and 51). It follows Nakajima and Flaks may be accordingly modified with the teachings of Lee to change its at least one time varying content anchor according to a relative time.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Nakajima and Flaks with the teachings of Lee. Lee is analogous art in dealing with a system and method for synthesizing information of a real world and a virtual world to present augmented reality (page 1/par. 5). Lee discloses its use of relative time is advantageous in determining motion of real world objects to appropriately present augmented reality (pages 3-4/par. 42 and 51). Consequently, a PHOSITA would incorporate the teachings of Lee into Nakajima and Flaks for determining motion of real world objects to appropriately present augmented reality. Therefore, claim 32 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
Claims 33 and 35 is rejected under 35 U.S.C. 103 as being unpatentable over Nakajima in view of Flaks further in view of Liu et al. (U.S. Patent Application Publication 2013/0044128 A1, hereinafter “Liu”).
For claim 33, depending on claim 30, Nakajima as modified by Flaks does not disclose a time span.
However, these limitations are well-known in the art as disclosed in Liu.
Liu similarly discloses a system and method for mixing imagery of a real world and a virtual world to present augmented reality (page 1/par. 1). Liu discloses its system determines motion of real world objects with respect to a time period as a time span (page 17/par. 169). It follows Nakajima and Flaks may be accordingly modified with the teachings of Liu to change its at least one time varying content anchor according to a time period as a time span.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Nakajima and Flaks with the teachings of Liu. Liu is analogous art in dealing with a system and method for mixing imagery of a real world and a virtual world to present augmented reality (page 1/par. 1). Liu discloses its use of a time period is advantageous in determining motion of real world objects to appropriately present augmented reality (page 17/par. 169). Consequently, a PHOSITA would incorporate the teachings of Liu into Nakajima and Flaks for determining motion of real world objects to appropriately present augmented reality. Therefore, claim 33 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
For claim 35, depending on claim 30, Nakajima as modified by Flaks discloses wherein the at least one time varying content anchor changes according to a path of the one or more identified objects (Liu similarly discloses a system and method for mixing imagery of a real world and a virtual world to present augmented reality (page 1/par. 1); Liu discloses its system determines motion of real world objects with respect to a time period as a time span where the real world objects may move along trajectories as paths (page 17/par. 169); and it follows Nakajima and Flaks may be accordingly modified with the teachings of Liu to change its at least one time varying content anchor according to a path of its one or more identified objects).
Claim 37 is rejected under 35 U.S.C. 103 as being unpatentable over Nakajima in view of Flaks further in view of Haritaoglu et al. (U.S. Patent Application Publication 2011/0122255 A1).
For claim 37, depending on claim 30, Nakajima as modified by Flaks does not disclose a contextual signature.
However, these limitations are well-known in the art as disclosed in Haritaoglu.
Haritaoglu similarly discloses a system and method for performing object recognition in captured video (page 5/par. 62). Haritaoglu explains objects are associated with contextual signatures for recognizing the objects and corresponding motion of the objects in the video (page 5/par. 67). It follows Nakajima and Flaks may be accordingly modified with the teachings of Haritaoglu to implement a contextual signature for its at least one time varying content anchor.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Nakajima and Flaks with the teachings of Haritaoglu. Haritaoglu is analogous art in dealing with a system and method for performing object recognition in captured video (page 5/par. 62). Haritaoglu discloses its use of a contextual signature is advantageous in appropriately recognizing an object and corresponding motion in captured video (page 5/par. 62 and 67). Consequently, a PHOSITA would incorporate the teachings of Haritaoglu into Nakajima and Flaks for appropriately recognizing an object and corresponding motion in captured video. Therefore, claim 37 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
Claim 44 are rejected under 35 U.S.C. 103 as being unpatentable over Nakajima in view of Flaks further in view of Douris et al. (U.S. Patent Application Publication 2009/0289956 A1, hereinafter “Douris”) (made of record of the IDS submitted 1/24/2025).
For claim 44, depending on claim 30, Nakajima as modified by Flaks does not disclose a content source comprises a commercial website.
However, these limitations are well-known in the art as disclosed in Douris.
Douris similarly discloses a system and method for presented augmented reality as virtual billboards with a reality overlay device (page 1/par. 5). Douris explains its virtual billboards comprise overlay images may include commercial advertisements of products that may be of interest to a user (page 4/par. 43, pages 5-6/par. 62 and pages 6-7/par. 74). Douris explains its virtual billboards for presenting advertisements may access its content from websites managed by businesses for customizing advertisements for presentation on the virtual billboards (page 7/par. 80-81). It follows Nakajima and Flaks may be accordingly modified with the teachings of Douris to present images of commercial products in its presentation of augmented reality.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Nakajima and Flaks with the teachings of Douris. Douris is analogous art in dealing with a system and method for presented augmented reality with a reality overlay device (page 1/par. 5). Douris discloses its use of virtual billboards is advantageous in presenting images of commercial products to tailor the presentation of augmented reality to match the preferences of a user (pages 6-7/par. 74-75). Consequently, a PHOSITA would incorporate the teachings of Douris into Nakajima and Flaks for presenting images of commercial products to tailor the presentation of augmented reality to match the preferences of a user. Therefore, claim 44 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
Claims 47 are rejected under 35 U.S.C. 103 as being unpatentable over Nakajima in view of Flaks further in view of Maizels et al. (U.S. Patent Application Publication 2013/0107021 A1, hereinafter “Maizels”) (made of record of the IDS submitted 1/24/2025).
For claim 47, depending on claim 30, Nakajima as modified by Flaks discloses an environmental model of the 3D environment (Flaks similarly discloses a system and method for positioning virtual image content with respect to corresponding real world objects to present augmented reality (page 1/par. 4) and explains its system generates a three dimensional environmental model for a 3D augmented reality environment for rendering the virtual image content at the object position and orientation within the model (page 6/par. 61, page 10/par. 102 and page 11/par. 108-111) and it follows Nakajima may be accordingly modified with the teachings of Flaks for generating an environmental model for its 3D augmented reality environment).
Nakajima as modified by Flaks does not disclose a detected object comprises a coordinate within a 3D environment.
However, these limitations are well-known in the art as disclosed in Maizels.
Maizels similarly discloses a system and method for positioning virtual image content with respect to corresponding real world objects to present augmented reality (page 1/par. 3 and page 2/par. 28). Maizels explains detected real world objects comprise coordinates within a 3D environment for registration with corresponding virtual content (page 2/par. 29; and page 5/par. 58 and 63). It follows Nakajima and Flaks may be accordingly modified with the teachings of Maizels for determining coordinates of its detected objects as its time varying content anchors within its environmental model for presenting its 3D augmented reality environment.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Nakajima and Flaks with the teachings of Maizels. Maizels is analogous art in dealing with a system and method for positioning virtual image content with respect to corresponding real world objects to present augmented reality (page 1/par. 3 and page 2/par. 28). Maizels discloses the determination of coordinates for detected real world objects is advantageous in ensuring appropriate registration of real world objects with corresponding virtual content for presenting augmented reality (page 2/par. 29; and page 5/par. 58 and 63). Consequently, a PHOSITA would incorporate the teachings of Maizels into Nakajima and Flaks for ensuring appropriate registration of real world objects with corresponding virtual content for presenting augmented reality. Therefore, claim 47 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHARLES TSENG whose telephone number is (571)270-3857. The examiner can normally be reached on 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHARLES TSENG/ Primary Examiner, Art Unit 2613
Prosecution Insights