DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/20/2024 and 09/24/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 1and 9 are objected to because of the following informalities: two hyphens (-) is included in the body of the claim for indentation. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the available information" in line 18. There is insufficient antecedent basis for this limitation in the claim. Claim 2-5 inherit their indefiniteness from which they depend.
Claim 9 recites the limitation “the available information” in line 15. Claim 10-11 inherit their indefiniteness from which they depend.
Claim 6 recites the limitation "the image" in line 9. It is ambiguous which renders the claim indefinite because it is unclear whether the limitation refers to “the image” in line 5 or “a video” in line 8. In particular, although claim 6 states “how the image should be integrated”, the specification suggests claimed step performs on the “a video.” See step 606 in FIG. 6; Specification, para. [0070].
Claim 7 recites “wherein the metadata describing how the image should be integrated into and augmented reality environment includes”. However, the examiner is unclear as to which image the claim is referencing. Is the claim referencing the image (step 603) or the image in the video (step 605)? Thus, claim 7 is unclear and indefinite.
Claim 8 inherits its indefiniteness from claim 6 from which it depends.
Claims 1-11 will be examined as best understood by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5 and 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over
Christensen et al. (US 2015058102 A1-IDS REF) (hereinafter Christensen) in view of Kraytem et al. (US 20190286646 A1) (hereinafter Kraytem), and further in view of Chowdhary et al. (US 20170039771 A1- IDS REF) (hereinafter Chowdhary).
Regarding claim 1, Christensen discloses A method in a computing device of providing an external message in an augmented reality environment, comprising:
(Abstract, "The disclosure includes a system and method for generating virtual reality content.")
receiving the external message including metadata relating to the desired presentation of the external message in the augmented reality environment;
(para. [0073], "In some embodiments, the ad server 141 receives metadata associated with virtual reality content displayed by the viewing system 133 and selects advertisements for presentation in conjunction with the virtual reality content based on the metadata. For example, the ad server 141 selects stored advertisements based on keywords associated with the virtual reality content."; para. [0138], “the advertisement data includes data describing how the advertisement may be incorporated in the virtual reality content. For example, the advertisement data describes where the advertisement may be included in the virtual reality content.")
integrating the external into the augmented reality environment based on a comparison of characteristics (Examiner’s note: advertisement module based on user input determines the advertisement) of the augmented reality environment with the metadata relating to the desired presentation of the external message in the augmented reality environment; and (para. [0138], "In some embodiments, the advertisement data includes data describing how the advertisement may be incorporated in the virtual reality content. For example, the advertisement data describes where the advertisement may be included in the virtual reality content. The advertising module 216 may analyze the advertisement data and incorporate the advertisement in the virtual reality content according to the advertisement data… the user 134 provides user input to the advertising module 216 and the user input specifies a user preference describing how the advertisement may be incorporated in the virtual reality content. The advertising module 216 may analyze the user input and incorporate the advertisement based at least in part on the user input.")
rendering the augmented reality environment with the external message integrated into the augmented reality environment; (para. [0137], “The advertising module 216 may be software including routines for adding advertisements to the virtual reality content generated by the content system 131.”)
wherein: the metadata relating to the presentation of the external message in the augmented reality environment includes technical criteria for integration of the external message into the augmented reality environment; and (para. [0039], “For example, the advertisements are displayed in areas that are unobtrusive, such as above the user or below the user. The virtual reality system may be able to determine how to charge for the advertisements based on a location of the user's gaze.”; para. [0073], "In some embodiments, the ad server 141 receives metadata associated with virtual reality content displayed by the viewing system 133 and selects advertisements for presentation in conjunction with the virtual reality content based on the metadata."; para. [0138], "In some embodiments, the advertisement data includes data describing how the advertisement may be incorporated in the virtual reality content. For example, the advertisement data describes where the advertisement may be included in the virtual reality content.” Examiner’s note: technical criteria is placement of advertisement based on user’s gaze or data included in the metadata determines selection criteria of VR content.)
the metadata relating to the presentation of the external message para. [0138], "In some embodiments, the advertisement data includes data describing how the advertisement may be incorporated in the virtual reality content. For example, the advertisement data describes where the advertisement may be included in the virtual reality content.”)
if the available metadata is sufficient for integration, integrating the external message into the augmented reality environment based on the available information, and (para. [0138], "In some embodiments, the advertisement data includes data describing how the advertisement may be incorporated in the virtual reality content. For example, the advertisement data describes where the advertisement may be included in the virtual reality content. The advertising module 216 may analyze the advertisement data and incorporate the advertisement in the virtual reality content according to the advertisement data”)
Christensen does not explicitly disclose the metadata
Kraytem teaches that a system for “Detection and enrichment of missing data or metadata for large data sets” (Title; Abstract) comprises the metadata (para. [0090], “FIG. 4 is a flow chart diagram illustrating functionality of the metadata enrichment system related to identification of one or more missing metadata item values and enrichment of the missing metadata item values, according to various embodiments”; para. [0099], “In some embodiments, the artificial intelligence may associate one or more identified likely values with confidence levels and may only make a suggestion when the confidence level is sufficiently met (such as having higher value than a threshold value).”)
As both Christensen and Kraytem are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the metadata in the context of augmented reality, by Christensen according to the teaching of Kraytem in order not to provide limited information based on missing metadata (Kraytem, para. [0005])
Christensen and Kraytem fails to specifically disclose -if the available metadata is insufficient for integration, integrating the external message into the augmented reality environment based on a process of surface detection (507) in the augmented reality environment and integration of the of the external message in conjunction with a detected surface in the augmented reality environment.
Chowdhary teaches that systems and methods for inserting contextual advertisements into a virtual environment” (Title, Abstract) comprise integrating the external message into the augmented reality environment based on a process of surface detection (507) in the augmented reality environment and integration of the external message in conjunction with a detected surface in the augmented reality environment. (claim 1, “injecting digital content by superimposing text, graphics, video content over at least a part of the key frame”; para. [0054], “A system and method is provided for injecting ads when encountering a flat feature rich surface in Augmented Reality (AR) space 101.”)
As Christensen, Kraytem and Chowdhary are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include integrating the external message into the augmented reality environment based on a process of surface detection (507) in the augmented reality environment and integration of the external message in conjunction with a detected surface in the augmented reality environment, in the context of augmented reality, as taught by the combination of Christensen and Kraytem, according to the teaching of Chowdhary, in order to provide the addition of a computer-assisted contextual layer of information over the real world, creating a reality that is enhanced or augmented (Chowdhary, para. [0005]).
Regarding claim 2, the combination of Christensen, Kraytem, and Chowdhary further teaches wherein the technical criteria for integration specifies at least one of the absolute position of the external message in the augmented reality environment, the position of the external message in the augmented reality environment relative to another object in the augmented reality environment, that the external message should be positioned on a surface in the augmented reality environment, and that the external message should be positioned on or near a specific object or type of object in the augmented reality environment. (Christensen, para. [0073], “In some embodiments, the ad server 141 receives metadata associated with virtual reality content displayed by the viewing system 133 and selects advertisements for presentation in conjunction with the virtual reality content based on the metadata”; para. [0138], “In some embodiments, the advertisement data includes data describing how the advertisement may be incorporated in the virtual reality content. For example, the advertisement data describes where the advertisement may be included in the virtual reality content.”)
Regarding claim 3, the combination of Christensen, Kraytem, and Chowdhary further teaches further comprising transmitting context information to a remote message server (403), and wherein: the step of receiving the external message includes (Christensen, para [0013], “The features may include the playlist being based on most user views of virtual reality content, a geographical location, or the playlist is generated by a user that is an expert in subject matter and the playlist is based on the subject matter.”
para. [0058], “The client device 127 may be configured to transmit data to the server 129 or to receive data from the server 129 via the network 105.”; para. [0070], “The server 129 sends and receives data to and from one or more of the other entities of the system 100 via the network 105…For example, the server 129 receives VR content including a stream of 3D video data (or compressed 3D video data) and a stream of 3D audio data (or compressed 3D audio data) from the client device 127 and stores the VR content on a storage device associated with the server 129.”)
receiving a set of messages (405) that have been selected based on a comparison of the transmitted context information with selection criteria associated with the respective external messages (Christensen, para. [0071], “the ad server 141 is an advertisement repository for advertisements that are requested by the content system 131 for display as part of the virtual reality content.”; para. [0072], “In some embodiments, the ad server 141 includes rules for targeting advertisements to specific users, for targeting advertisements to be displayed in conjunction with various types of content)
and adding the received messages to local storage. (Christensen, para. [0058], “The client device 127 may be configured to transmit data to the server 129 or to receive data from the server 129 via the network 105.”; para. [0070], “The server 129 sends and receives data to and from one or more of the other entities of the system 100 via the network 105…For example, the server 129 receives VR content including a stream of 3D video data (or compressed 3D video data) and a stream of 3D audio data (or compressed 3D audio data) from the client device 127 and stores the VR content on a storage device associated with the server 129.”)
Regarding claim 4, the combination of Christensen, Kraytem, and Chowdhary further teaches wherein the external message is selected from the set of messages based on prioritization according to a predetermined scoring system for prioritization of messages (501). (Christensen, para. [0071], “the ad server 141 is an advertisement repository for advertisements that are requested by the content system 131 for display as part of the virtual reality content.”; para. [0072], “In some embodiments, the ad server 141 includes rules for targeting advertisements to specific users, for targeting advertisements to be displayed in conjunction with various types of content (e.g., content served by the content server 139, virtual reality content served by the client device 127 or the server 129), for targeting advertisements to specific locations or Internet Protocol (IP) addresses associated with the client device 127, the viewing system 133, or the user 134. The ad server 141 may include other rules for selecting and/or targeting advertisements.”; para. [0073], “In some embodiments, the ad server 141 receives metadata associated with virtual reality content displayed by the viewing system 133 and selects advertisements for presentation in conjunction with the virtual reality content based on the metadata. For example, the ad server 141 selects stored advertisements based on keywords associated with the virtual reality content. Other methods are possible for providing targeted advertisements to users, which may alternatively or additionally be implemented in the embodiments described herein.”)
Regarding claim 5, the combination of Christensen, Kraytem, and Chowdhary further teaches wherein the metadata relating to the presentation of the external message in the augmented reality environment includes selection criteria specifying at least one of information about loaded augmented reality content, a feature detected in the augmented reality environment, a device location, and a user demographic. (Christensen, para. [0072], “In some embodiments, the ad server 141 includes rules for targeting advertisements to specific users, for targeting advertisements to be displayed in conjunction with various types of content (e.g., content served by the content server 139, virtual reality content served by the client device 127 or the server 129), for targeting advertisements to specific locations or Internet Protocol (IP) addresses associated with the client device 127, the viewing system 133, or the user 134. The ad server 141 may include other rules for selecting and/or targeting advertisements.”)
Regarding claim 9, Christensen discloses A device (see figure 1) configured to present an augmented reality environment and including an augmented reality app module (302) (Christensen, para. [0061], “The viewing system 133 may include or use a computing device to decode and render a stream of 3D video data on a virtual reality display device (e.g., Oculus Rift virtual reality display) or other suitable display devices that include, but are not limited to: augmented reality glasses; televisions, smartphones, tablets, or other devices with 3D displays and/or position tracking sensors; and display devices with a viewing position control, etc” Examiner’s note: listed devices would include app module to implement virtual or augmented reality environment.) configured to receive external messages, a camera module (see figure 1 of Christensen; paragraphs 41, 60 97, and 111) configured to receive image information relating to a local scene from a device camera, and an augmented reality content integration module (see figure 1 of Christensen) configured to create an augmented reality environment by receiving and integrating images from the camera module (305), main augmented reality content from local memory (see also paragraphs 32 and 49), and external messages from the augmented reality app module (302); wherein: the augmented reality integration module (306) is configured to integrate the external messages into the augmented reality environment based on metadata associated with the respective messages and including technical criteria for integration of the external message into the augmented reality environment ; and the augmented reality app module (302) is configured to evaluate the metadata relating to the presentation of the external message for completeness with respect to integration (502)- the claimed integration module and augmented reality app module include similar limitations as discussed in claim 1, except claim 9 is a system claim, as exhibited in Christensen figure 1, therefore the limitations are rejected under similar rationale as claim 1-see supra rejection of claim 1)
PNG
media_image1.png
573
810
media_image1.png
Greyscale
Regarding claim 10, claim 10 has similar limitations as in claim 2, except it is a system claim, therefore it is reasoned as discussed in claim 2.
Regarding claim 11, the combination of Christensen, Kraytem, and Chowdhary further teaches further configured to receive main augmented reality content from an augmented reality content service (303) and external messages from a message service (304). (Christensen, Figure 1; para. [0072], “content served by the content server 139, virtual reality content served by the client device 127 or the server 129")
Claims 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Christensen et al. (US 20150058102 A1- IDS REF) (hereinafter Christensen) in view of Hori et al. (US 20220114367 A1) (hereinafter Hori).
Regarding claim 6, Christensen discloses a method in a server of generating an external message that may be integrated into an augmented reality environment, comprising: determining if the message is an image (603), and if it is, generating and embedding metadata describing how the image should be integrated into an augmented reality environment (604); determining if the message is a video (605), and if it is, generating and embedding metadata describing how the image should be integrated into an augmented reality environment (606); and storing or transmitting the message as an external augmented reality message (608). (para. [0072], “for targeting advertisements to be displayed in conjunction with various types of content (e.g., content served by the content server 139, virtual reality content served by the client device 127 or the server 129); para. [0137], “For example, the ad server 141 stores and transmits advertisement data that describes one or more advertisements. The advertising module 216 incorporates the advertisements in the virtual reality content. For example, the advertisement includes an image, audio track, or video, and the advertising module 216 incorporates the advertisement into the virtual reality content…. In some embodiments, the advertisements from the ad server 141 include ad data describing a location for displaying the advertisement.")
Christensen does not explicitly disclose a determining if the message is pure text (601), and if it is, converting the text to an image (602);
However, Hori more explicitly teaches determining if the message is pure text (601), and if it is, converting the text to an image (602); (para. [0171], “In response to detecting that text is selected and dragged, the visual communication system 4 converts the selected and dragged text data into image data.”)
As both Christensen and Hori are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include determining if the message is pure text (601), and if it is, converting the text to an image (602), in the context of augmented reality, by Christensen according to the teaching of Hori in order for users to control augmented reality presentation more effectively and efficiently.
Regarding claim 7, the combination of Christensen and Hori further teaches wherein the metadata describing how the image should be integrated into an augmented reality environment includes technical criteria specifying at least one of the absolute position of the external message in the augmented reality environment, the position of the external message in the augmented reality environment relative to another object in the augmented reality environment, that the external message should be positioned on a surface in the augmented reality environment, and that the external message should be positioned on or near a specific object or type of object in the augmented reality environment. (Christensen, para. [0073], “In some embodiments, the ad server 141 receives metadata associated with virtual reality content displayed by the viewing system 133 and selects advertisements for presentation in conjunction with the virtual reality content based on the metadata”; para. [0138], “In some embodiments, the advertisement data includes data describing how the advertisement may be incorporated in the virtual reality content. For example, the advertisement data describes where the advertisement may be included in the virtual reality content.”)
Regarding claim 8, the combination of Christensen and Hori further teaches further comprising embedding metadata relating to the selection of the external message for integration in the augmented reality environment, including selection criteria specifying at least one of information about loaded augmented reality content, a feature detected in the augmented reality environment, a device location, and a user demographic. (Christensen, para. [0072], “In some embodiments, the ad server 141 includes rules for targeting advertisements to specific users, for targeting advertisements to be displayed in conjunction with various types of content (e.g., content served by the content server 139, virtual reality content served by the client device 127 or the server 129), for targeting advertisements to specific locations or Internet Protocol (IP) addresses associated with the client device 127, the viewing system 133, or the user 134. The ad server 141 may include other rules for selecting and/or targeting advertisements.”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hyorim Park/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615