DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments filed 2/19/2026, with respect to claims 1-21 have been fully considered but are moot in view new ground(s) of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 8, 10, 11, 13 and 16-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grauer (PGPUB Document No. US 2017/0256040) in view of Grauer (PGPUB Document No. US 2017/0256040) in view of Tran et al. (PGPUB Document No. US 2017/0323481) in view of Haro (PGPUB Document No. US 2022/0277565) in view of Stump et al. (PGPUB Document No. US 2021/0096543) in view of Krueger (PGPUB Document No. US 2024/0020919) in view of Shtok et al (PGPUB Document No. US 2022/0277524).
Regarding claim 1, Grauer teaches a method comprising:
Monitoring an environment via one or more sensors of a computing device to generate event data, the one or more sensors comprising at least a three-dimensional video sensor to generate three-dimensional video data (camera acquiring 3D image frames (video) (Grauer: 0020)), and a location sensor to generate location data (location measurement unit (LMU) 114 determining metadata such as the current location (GPS) and orientation of a scene capture device (Grauer: 0022, 0030). Note, GPS data correspond to environment data);
Wherein the event data comprises the location data that indicates a recording location of the event data (Each database image may be linked to associated data and parameters, such as the location coordinates and the viewing angle of the imaged scene (Grauer: 0023)) a stored video recording previously recorded in the environment (the database images may be continuously updated in real-time. For example, different users of system 100 may be authorized to upload new images to database 110, and/or to delete or modify existing images (Grauer: 0023)),
Detecting, based on a trigger that comprises a comparison of the event data against memory experience data within a data store to detect a match, that event data related to the environment (retrieved data from the database 0030, wherein said data may be user created content as disclosed in 0044 of Grauer, wherein retrieving content with matching location data requires comparing of data as presently claimed), wherein the event data comprises the location data that indicates a recording location of the event data (Each database image may be linked to associated data and parameters, such as the location coordinates and the viewing angle of the imaged scene (Grauer: 0023)), a stored video recording previously recorded in the environment, and three-dimensional video data (the database comprise of 3D models (Grauer: 0023));
Identifying one or more participants in the event data (the determined position and orientation information of a camera (Grauer: 0022), the event comprising the user creating the content and a second person that is designated by the user to share the content (Grauer: 0041));
Generating, based on the event data, augmented reality event data, virtual reality event data, or a combination thereof (creating user content according to various teachings disclosed in FIG.6 of Grauer (Grauer: 0042-0048), wherein the content maybe augmented reality (Grauer: 0035))
And presenting the augmented reality event data, the virtual reality event data, or a combination thereof, to the one or more participants (other users viewing the shared content (Grauer: 0042)).
However, Grauer does not expressly teach the 3D video sensor being a 3D volumetric video sensor. Stump teaches the concept of recording 3D volumetric videos for VR applications (Stump: 0069)). Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to record the interactions as 3D volumetric videos (taught by Stump), because this enables effective playback of 3D content within a VR environment.
Further, the combined teachings do not expressly teach but Tran teaches
The augmented reality event data that comprises representations of the one or more participants identified within the event data (communicating while displaying virtual representations of friends (Tran: 0256)),
The augmented reality event data comprising at least one holographic display of a graphical representation of one or more remote participants presented at an augmented reality device during synchronous presentation of the event data at a plurality of augmented reality devices (render the virtual image of his/her friends on a display interface such as holographic display interface, wherein the user is able to communicate with friends while dining (real-time, synchronous) (Tran: 0256)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the holographic display of Tran, because this enables sharing AR experiences with friends.
And further, the combined teachings above do not expressly teach but Haro teaches,
The at least one holographic display of the first participant of the one or more participants being presented at the computing device of the second participant, the second participant experiencing the at least one holographic display of the first participant from a perspective of the first participant (Haro teaches an immersive reality system that generate and provide/display multiple unique perspectives of each user/participant to each other user/participant by feeding their real-time video (Haro: 0045). Therefore, Haro teaches the concept of each participant being able to visually see each other as they interact with one another (Haro: 0042). Utilizing the teachings of Haro enables a two holographic AR communication system where each participant are able to view each other’s holographic representations).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to view a virtual representation of each other (taught by Haro), because this enable an improved immersive experience.
And further, the combined teachings do not expressly teach but Krueger teaches wherein the generating the augmented reality event data comprises processing the three-dimensional video data to extract one or more three-dimensional objects from two-dimensional frames (Krueger teaches the concept of using photogrammetry to extract 3D objects from 2D images (Krueger: 0027). Applying the teachings of Krueger enables the database of 3D models of Grauer to include 3D models obtained using the teachings of Krueger).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such that 3D models are obtained in the manner taught by Krueger, because this enables an effective method of obtaining 3D objects from 2D images.
And further, the combined teachings above do not expressly teach but Shtok teaches wherein the perspective of the first participant is based on sensor data from the one or more sensors of the computing device (Shtok teaches the concept of anchoring the pose/position of the expert to the user (Shtok: 0045, 0044), wherein the pose/position is based one or more sensors (Shtok: 0022)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the sensor teachings of Shtok, because this provides an effective method of perspective alignment during social memory re-experiencing.
Regarding claim 8, the combined teachings above teach the method of claim 1, wherein applying the trigger to detect that the event data is stored comprises comparing a trigger data associated with the trigger against the event data to determine that a match exists (identifying matching metadata in the database (Grauer: 0038, 0039)).
Regarding claim 10, the combined teachings above teach the method of claim 1, wherein the one or more sensors are disposed in a head-wearable apparatus, in a drone, in a mobile device, in a portable computing device, or a combination thereof (camera within a head mounted apparatus (Grauer: 0020)).
Regarding claim 11, the combined teachings above teach the method of claim 1, wherein the one or more sensors comprise a sound sensor, a camera sensor, a weather sensor, a light condition sensor, a location sensor, a microlocation sensor, a smell sensor, a taste sensor, a biosignal sensor, or a combination thereof (camera (Grauer: 0020)).
Claim(s) 13 and 16 are corresponding system claim(s) of claim(s) 1 and 10. The limitations of claim(s) 13 and 16 are substantially similar to the limitations of claim(s) 1 and 10. Therefore, it has been analyzed and rejected substantially similar to claim(s) 13 and 16. Note, Grauer teaches one or more hardware processors (multiple processing units (Grauer: 0026)), at least one memory (memory (Grauer: 0026)), as presently claimed.
Regarding claim 17, the combined teachings above teach the system of claim 13, wherein the at least one memory stores instructions that cause the one or more hardware processors to perform operations comprising:
recording a new event data by capturing data comprising a picture, a video, a sound, a smell signal, a taste signal, a weather condition, a location, a microlocation, a lighting condition, a person, a time, a date, a pet, a social network, a biosignal, or a combination thereof, via the one or more sensors (image and/or video captured by a camera (Grauer: 0020));
Storing the new event data in the data store (database coupled with the camera for storing image information (Grauer: 0023));
And indexing the new event data, wherein the indexing comprises relating the data to a second trigger (database 110 may include metadata (i.e., a “tag”), for assisting identification and classification of the images, and subsequent browsing and searching by different users based on relevant image content (Grauer: 0038). Note the second trigger corresponds to metadata newly formed by the new recording).
Regarding claim 18, the combined teachings above teach the system of claim 17, wherein the at least one memory stores instructions that cause the one or more hardware processors to perform operations comprising:
Applying the second trigger to detect that the new event data is stored in the data store based on the monitoring (retrieving content based on the metadata (Grauer: 0010)).
Claim(s) 19 is a corresponding machine-readable medium claim(s) of claim(s) 1. The limitations of claim(s) 19 are substantially similar to the limitations of claim(s) 1. Therefore, it has been analyzed and rejected substantially similar to claim(s) 19. Note, Grauer teaches a machine-readable medium (memory or storage unit (Grauer: 0026)).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grauer in view of Tran in view of Haro in view of Stump in view of Krueger in view of Shtok as applied to claim(s) above, and further in view of Ding et al. (PGPUB Document No. US 2020/0260149).
Regarding claim 2, the combined teachings above does not expressly teach but Ding teaches the method of claim 1, wherein presenting the augmented reality event data, to the one or more participants in the synchronous mode comprises presenting the augmented reality event data, so that at least two of the one or more participants interact with the augmented reality event data at a same time (Ding teaches the concept of user sharing content synchronously with other users (Ding: 0093)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the teachings of Ding, because this enables an effective method of sharing AR contents to other users.
Claim(s) 3 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grauer in view of Tran in view of Haro in view of Stump in view of Krueger in view of Shtok as applied to claim(s) above, and further in view of Rogers et al. (PGPUB Document No. US 2019/0138260).
Regarding claim 3, the combined teachings above does not expressly teach but Rogers teaches the method of claim 1, wherein presenting the augmented reality event data, to the one or more participants in the asynchronous mode comprises presenting the augmented reality event data, so that at least two of the one or more participants interact with the augmented reality event data, at a different time (Rogers teaches the concept of sharing user created AR objects using a subscription method, which corresponds to the AR object being shared asynchronously (Rogers: 0042, FIG.5)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the subscription teaching of Rogers, because this enables an effective method of sharing AR contents to other users.
Regarding claim 12, the combined teachings above does not expressly teach but Roger teaches the method of claim 1, wherein presenting the augmented reality event data, comprises
presenting the augmented reality event data only to participants of the one or more participants that have communicated an acceptance of an invitation to interact with the augmented reality event data (other users can access shared content only through a subscription (Rogers: 0042)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the subscription teaching of Rogers, because this enables an effective method of sharing AR contents to other users.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grauer in view of Tran in view of Haro in view of Stump in view of Krueger in view of Shtok as applied to claim(s) above, and further in view of Weinheimer et al. (PGPUB Document No. US 2021/0142578).
Regarding claim 9, the combined teachings above teaches the method of claim 8, wherein the trigger data comprises a location, at least one participant of the one or more participants, a picture, a video, a sound, a biosignal, a smell signal, a taste signal, a weather condition, a lighting condition, a time, a date, a person, a pet, or a combination thereof (metadata comprise of group (“at least one participant”), time and location information (Grauer: 0039)).
However, Grauer does not expressly teach but Weinheimer teaches the trigger data comprising a a microlocation (Weinheimer: 0096). Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to further include microlocation as taught by Weinheimer, because this enables taking advantage of the benefits of BLE.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grauer in view of Tran in view of Haro in view of Stump in view of Krueger in view of Shtok as applied to claim(s) above, and further in view of Ding in view of Rogers.
Regarding claim 15, the combined teachings above does not expressly teach but Ding teaches the system of claim 13, wherein presenting the augmented reality event data, to the one or more participants in the synchronous mode comprises presenting the augmented reality event data, so that at least two of the one or more participants interact with the augmented reality event data, at a same time (Ding teaches the concept of user sharing content synchronously with other users (Ding: 0093)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the teachings of Ding, because this enables an effective method of sharing AR contents to other users.
However, the combined teachings does not expressly teach but Rogers teaches wherein presenting the augmented reality event data, to the one or more participants in the asynchronous mode comprises presenting the augmented reality event data, so that at least two of the one or more participants interact with the augmented reality event data, at a different time (Rogers teaches the concept of sharing user created AR objects using a subscription method, which corresponds to the AR object being shared asynchronously (Rogers: 0042, FIG.5)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the subscription teaching of Rogers, because this enables an effective method of sharing AR contents to other users.
Claim(s) 4-6, 14 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grauer in view of Tran in view of Haro in view of Stump in view of Krueger in view of Shtok as applied to claim(s) above, and further in view of Moran (PGPUB Document No. US 2018/0130257).
Regarding claim 4, the combined teachings above does not expressly teach but Moran teaches the method of claim 1, wherein presenting the augmented reality event data, to the one or more participants in the asynchronous mode comprises
recording a video, recording a picture, or a combination thereof, of a user of a plurality of users interacting with the augmented reality event data, during the presenting (Moran teaches the concept of recording interactions for later playback on VR devices (Moran: 0053, 0010, 0006), wherein the interaction may be interaction with varying aspects with a virtual location (“interacting with the augmented reality data”)),
and communicating the recording, the picture, or the combination thereof, to a participant of the one or more participants, to an entity, or to a combination thereof (playback of the recorded content (Moran: 0053)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to enable recording of VR interactions, because this enables effective sharing of content with other users.
Regarding claim 5, the combined teachings above does not expressly teach but Moran teaches the method of claim 1, wherein presenting the augmented reality event data, to the one or more participants in the asynchronous mode comprises
recording a new event data of a user of a plurality of users interacting with the augmented reality event data, during the presenting (Moran teaches the concept of recording interactions for later playback on VR devices (Moran: 0053, 0010, 0006), wherein the interaction may be interaction with varying aspects with a virtual location (“interacting with the augmented reality data”). Note, the Examiner construes the recording that can be played back by other VR users as the “new event data,” as presently claimed),
and communicating the new event data to a participant of the one or more participants, to an entity, or to a combination thereof (playback of the recorded content (Moran: 0053)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to enable recording of VR interactions, because this enables effective sharing of content with other users.
Regarding claim 6, the combined teachings above does not expressly teach but Moran teaches the method of claim 1, wherein creating, based on the event data, the augmented reality event data, comprises
transforming the event data into an augmented reality augmentation, a virtual reality augmentation, an avatar, an augmented reality object, a virtual reality object (the VR recording interactions for later playback on other VR devices as stated in the rejection above (Moran: 0053, 0010, 0006), a smell, a taste, a haptic force, an Internet-of-Things (IoT) device command, a video, a picture, or a combination thereof.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to enable recording of VR interactions, because this enables effective sharing of content with other users.
Claim 14 is similar in scope to claims 4 and 5.
Claim 20 is similar in scope to claims 4 and 5.
Claim(s) 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grauer in view of Tran in view of Haro in view of Stump in view of Krueger in view of Shtok as applied to claim(s) above, and further in view of Jin et al. (PGPUB Document No. US 2019/0051046).
Regarding claim 21, the combined teachings teach the method of claim 1, wherein processing the three-dimensional volumetric video data to extract one or more three-dimensional objects from two-dimensional frames comprises (refer to the photogrammetry teachings of Krueger as applied to the rejection to claim 1 above).
However, the combined teachings do not expressly teach but Jin teaches the concept of,
synchronizing video frames from a plurality of camera sensors (matching video frame to the same time (Jin: 0129));
detecting overlapping regions in the two-dimensional frames (matching of frames to find common features from the video frames (Jin: 0129));
detecting a same feature in two or more of the two-dimensional frames (finding common features from the video frames (Jin: 0129));
and extracting the three-dimensional object based on detecting the same feature and the overlapping regions (The matched features are then processed through triangulation and 3D reconstructing process to be vertexes of the 3D model (Jin: 0129)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above, because this is a well known method of effectively carrying photogrammetry.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616
Prosecution Insights