DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
This application is a continuation of US Application No. 18/092,711, filed January 03,2023, which is a continuation of US Application No. 17/333,759, filed May 28, 2021, which is a continuation of US Application No. 16/696,596, filed November 26, 2019, which is a continuation of US Application No. 15/856,941, filed December 28, 2017.
Preliminary Amendment
The preliminary amendment submitted on 03/10/2025 is acknowledged.
Claim Objections
Claim 10 is objected to because of the following informalities: In claim 10, lines 2-3, “wherein the audio input is transcribed to a text input, and wherein the text input is provided for display to the at least at least one of the first avatar or the second avatar” should be corrected to “wherein the audio input is transcribed to a text input, and wherein the text input is provided for display to the at least . Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 14-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Valdivia et al. (US Publication Number 2018/0095636 A1, hereinafter “Valdivia”).
(1) regarding claim 2:
As shown in fig. 1, Valdivia disclosed a method (para. [0074], note that FIG. 1 illustrates an example network environment 100 associated with a social-networking system) comprising:
generating a 360-degree video, wherein the 360-video is provided to a plurality of devices of a plurality of users (para. [0111], note that the visual media items may be spherical or otherwise immersive in nature (e.g., 360-degree visual media items), the plurality of devices comprising a first device and a second device, and the plurality of users comprising a first user and a second user (para. [0077], note that a client system 130 may enable a network user at client system 130 to access network 110. A client system 130 may enable its user to communicate with other users at other client systems 130.);
determining that the first device of the first user is providing the 360-degree video to the first user from a first perspective (para. [0107], note that the virtual reality system may receive one or more inputs from an input device that specify an intent by the user to view a particular region of the virtual space. Also see, para. [0137], note that a user, upon viewing a 360 photo, may notice that there is a hint displayed at the right of the user's current field of view (e.g., an indication of a laughing icon with an arrow to the right));
causing a first avatar representing the first user to be overlaid at a first position of the 360-degree video associated with the first perspective (para. [0135], note that a user may be able to add text, images, etc., to the speech bubble, and put it over the head of the user's avatar (or another user's avatar, or any other suitable position in the virtual room));
determining that the second device of the second user is providing the 360-degree video to the second user from a second perspective (para. [0215], note that the computing system may send information configured to render a first user-interface element comprising a representation of the first social-commentary item and a second user-interface element indicating a presence of the second social-commentary item. At step 4930, the computing system may receive an input from the user comprising a request to change the first view to a second view of the virtual space); and
based at least in part on input received from at least one of the plurality of users, causing output of an interaction with at least one of the first avatar or the second avatar (para. [0151], note that the virtual room may include an “interactive surface,” which may be a specific region in the virtual room having special properties. In particular embodiments, the interactive surface may resemble a table, desk, or other such surface that may be visible to all users in the virtual room. In particular embodiments, the interactive surface may afford users within the virtual room a means of sharing in certain interactive experiences. In particular embodiments, the interactive surface may be a means for users to share content with other users in the virtual room. The content that is being shared (e.g., a slide show, a video), may be two-dimensional or three-dimensional, and may be flat or non-flat content (e.g., spherical content, 360-degree content, 180-degree content)).
Valdivia disclosed most of the subject matter as described as above except for specifically teaching causing a second avatar representing the second user to be overlaid at the second position of the 360-degree video associated with the second perspective.
However, it would have been obvious for Valdivia to teach causing a second avatar representing the second user to be overlaid at the second position of the 360-degree video associated with the second perspective (para. [0146], note that the virtual reality system may render avatars of the users within the virtual room. An avatar in the virtual room may be a customizable generated rendition of the respective user. In particular embodiments, the virtual space may render a video-representation of the user (e.g., captured from a camera directed at the respected user). In particular embodiments, the rendered avatar may include one or more elements of the video-representation. As an example, the face of the avatar may be a face composed from the face in the video-representation).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art for Valdivia to teach causing a second avatar representing the second user to be overlaid at the second position of the 360-degree video associated with the second perspective. The suggestion/motivation for doing so would have been in order to provide a method of using controllers to interact with users in the virtual space (abs.). Therefore, it would have been obvious for Valdivia to obtain the invention as specified in claim 2.
(2) regarding claim 14:
Valdivia further disclosed the method of claim 2, further comprising:
receiving a request from the first device of the first user to switch from the first position of the 360-degree video to the second position of the 360-degree video (para. [0148], note that the user may be able to switch out existing features for a different feature that may be selected from a group of possible features); and
based at least in part on the received request, causing the first avatar representing the first user to be overlaid at the second position of the 360-degree video associated with the second perspective (para. [0106], note that the virtual space may be an augmented reality space in which virtual elements are overlaid on the real world. As an example, the virtual reality system may continuously capture images of the real world (e.g., using a camera on the headset of the user) and overlay virtual objects or avatars of other users on these images, such that a user may interact simultaneously with the real world and the virtual world).
(3) regarding claim 15:
Valdivia further disclosed the method of claim 2, further comprising: causing the second avatar to be overlaid at a third position of the 360-degree video proximate to the first position of the first avatar (para. [0100], note that objects that are geographically closer to each other may be considered to be more related or of more interest to each other than more distant objects. In particular embodiments, the coefficient of a user towards a particular object may be based on the proximity of the object's location to a current location associated with the user (or the location of a client system 130 of the user). A first user may be more interested in other users or concepts that are closer to the first user).
(4) regarding claim 16:
Valdivia further disclosed the method of claim 2, wherein the 360-degree video is virtual reality media asset (para. [0015], note that the virtual reality system may enable users to alter and share content items (e.g., photos/videos) in a virtual space).
The proposed rejection of claim 2, renders obvious the system claim 17, see fig. 1, because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claim 2 is equally applicable to claim 17.
Claim(s) 3-9, 10-11, 18-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Valdivia et al. (US Publication Number 2018/0095636 A1, hereinafter “Valdivia”) in view of Crutchfield et al. (US Publication Number 2017/0359690 A1).
(1) regarding claim 3:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching wherein the 360-degree video is a live sporting event.
However, Crutchfield disclosed wherein the 360-degree video is a live sporting event (para. [0008], note that methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc. Also see, para. [0065] live scanning of user’s surrounding in 3D).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the 360-degree video is a live sporting event. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 3.
(2) regarding claim 4:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching wherein causing the second avatar representing the second user to be overlaid at the second position of the 360-degree video associated with the second perspective is performed based at least in part on identifying the second user as a friend of the first user.
However, Crutchfield disclosed wherein causing the second avatar representing the second user to be overlaid at the second position of the 360-degree video associated with the second perspective is performed based at least in part on identifying the second user as a friend of the first user (para. [0079], note that the augmented reality module may overlay a representation 430/FIG. 22 (e.g., icon or photograph, plus name, location and/or distance) of each group member over a point in the scope image 432 having the position coordinates (e.g., latitude and longitude)).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein causing the second avatar representing the second user to be overlaid at the second position of the 360-degree video associated with the second perspective is performed based at least in part on identifying the second user as a friend of the first user. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 4.
(3) regarding claim 5:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching wherein identifying the second user as a friend of the first user is based at least in part on an indication of a relationship between the first user and the second user on a social network platform.
However, Crutchfield disclosed wherein identifying the second user as a friend of the first user is based at least in part on an indication of a relationship between the first user and the second user on a social network platform (para. [0057], note that the user may be asked to log in (Step 256) using the login credentials of one of user's social media sites (e.g., Facebook™), to log in by email, or to create login credentials in the application itself. Next, the user may be offered to add contacts (e.g., friends, family members, professional colleagues, etc) to the application, from user's social media sites (e.g., friends from Facebook™), or from the contacts stored on the user's mobile device, for example (step 256)).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein identifying the second user as a friend of the first user is based at least in part on an indication of a relationship between the first user and the second user on a social network platform. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 5.
(4) regarding claim 6:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching wherein the interaction is a reaction by at least one of the first avatar or the second avatar.
However, Crutchfield disclosed wherein the interaction is a reaction by at least one of the first avatar or the second avatar (para. [0082], note that freezing the screen may be useful such as when the user may want to start an interaction with a group member showing up in scope as the freezing may stop any jumping that may come from the user's camera movement. For example, tapping a group member's icon 433 i.e. user representation/avatar (see FIG. 24) may cause the expansion of that group member's container 434 (FIG. 25)).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the interaction is a reaction by at least one of the first avatar or the second avatar. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 6.
(5) regarding claim 7:
Valdivia further disclosed the method of claim 6, wherein the reaction by the at least one of the first avatar or the second avatar is generated for display on a hologram of the at least one of the first avatar or the second avatar (para. [0179], note that these notification elements may include the image element 3030 (e.g., a profile picture/video of the user who is sending the communication request, a live-streaming video or other representation of the user who is sending the communication request, etc.) and the text-notification element 3030. In this example, along with the notification elements).
(5) regarding claim 8:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching wherein the received input comprises an audio input received from the first user, wherein the audio input is provided for output from the first user to the at least one of the first avatar or the second avatar.
However, Crutchfield disclosed wherein the received input comprises an audio input received from the first user, wherein the audio input is provided for output from the first user to the at least one of the first avatar or the second avatar (para. [0080], note that e user could start a social media interaction (e.g., text messages, video or audio chat, etc.) with one or more group members (including simultaneously) appearing in scope).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the received input comprises an audio input received from the first user, wherein the audio input is provided for output from the first user to the at least one of the first avatar or the second avatar. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 8.
(6) regarding claim 9:
Valdivia further disclosed the method of claim 8, further comprising:
generating for display a hologram of the first user at the first position (para. [0177], note that referencing FIGS. 29A-29B, a particular user may select the spherical content 2910 for sharing with other users in the illustrated virtual room);
retrieving a voice sample of the first user (para. [0117], note that the user may be able to use speech input (e.g., using voice commands) to perform some of the same functions described herein in the context of gaze inputs); and
providing for output the audio input from the first user to the at least one of the first avatar or the second avatar via the hologram based at least in part on the voice sample of the first user (para. [0136], note that the audio-commenting tool, when selected and held, may function like a recording device that records the users voice and creates an audio-comment file that may be associated with the virtual space or content in the virtual space. The user (or other users with permission) may later access and play back the audio-comment file).
(7) regarding claim 10:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching wherein the received input comprises an audio input received from the first user, wherein the audio input is transcribed to a text input, and wherein the text input is provided for display to the at least at least one of the first avatar or the second avatar.
However, Crutchfield disclosed wherein the received input comprises an audio input received from the first user, wherein the audio input is transcribed to a text input, and wherein the text input is provided for display to the at least at least one of the first avatar or the second avatar (para. [0060], note that the user may then select (step 263) a group member's profile icon appearing in scope, to view (step 264) that group member's profile (e.g., name, age, etc) and/or to start (step 265) a text, audio, or video chat, or to exchange media (e.g., pictures) with that group member).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the received input comprises an audio input received from the first user, wherein the audio input is transcribed to a text input, and wherein the text input is provided for display to the at least at least one of the first avatar or the second avatar. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 10.
(8) regarding claim 11:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching retrieving an image of the first user from a user profile associated with the first user; extracting an image of the face of the first user from the image of the first user; and stitching the image of the face of the first user onto the first avatar.
However, Crutchfield disclosed retrieving an image of the first user from a user profile associated with the first user (para. [0072], note that the user's name and/or photograph may be retrieved from her Facebook™ page); extracting an image of the face of the first user from the image of the first user (para. [0110], note that users may assign certain preloaded images of graphics to individuals, which may be bundled or preloaded with the downloading of the application to the mobile device); and stitching the image of the face of the first user onto the first avatar (para. [0079], note that the augmented reality module may overlay a representation 430/FIG. 22 (e.g., icon or photograph, plus name, location and/or distance) of each group member over a point in the scope image).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach retrieving an image of the first user from a user profile associated with the first user; extracting an image of the face of the first user from the image of the first user; and stitching the image of the face of the first user onto the first avatar. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 11.
(9) regarding claim 13:
Valdivia disclosed most of the subject matter as described as above except for specifically teaching wherein the 360-degree video is provided to the first user from the first perspective based at least in part on a user selection, received from the first device of the first user, of the first perspective from the plurality of perspectives.
However, Crutchfield disclosed wherein the 360-degree video is provided to the first user from the first perspective based at least in part on a user selection, received from the first device of the first user, of the first perspective from the plurality of perspectives (para. [0065], note that users can easily scan (e.g., left or right) and view the location of other application groups/users, in real time and 3D space on the screen of their handheld device via augmented reality).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the 360-degree video is provided to the first user from the first perspective based at least in part on a user selection, received from the first device of the first user, of the first perspective from the plurality of perspectives. The suggestion/motivation for doing so would have been in order to provide systems and methods are disclosed herein which are designed to visually connect groups of friends, families and colleagues in large crowded environments like music festivals, theme parks, sporting events, conventions, etc (para. [0008]). Therefore, it would have been obvious to combine Valdivia with Crutchfield to obtain the invention as specified in claim 13.
The proposed rejection of claims 3-6, renders obvious the system claims 18-21, see fig. 1 because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claims 3-6 are equally applicable to claims 18-21.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Valdivia et al. (US Publication Number 2018/0095636 A1, hereinafter “Valdivia”) in view of Lanier et al. (US Patent Number 9,818,228 B2, hereinafter “Lanier”).
(1) regarding claim 12:
Valdivia further disclosed the method of claim 2, wherein the generating the 360-degree video further comprises:
receiving depth data from a plurality of cameras (para. [0108], note that the user's gaze may be tracked by one or more devices located remotely (e.g., one or more cameras or other sensors pointed toward the user that track the head and/or pupils of the user)).
Valdivia disclosed most of the subject matter as described as above except for specifically teaching based at least in part on the received depth data, generating a point cloud; and based at least in part on the point cloud, generating a plurality of perspectives from which the 360-degree video is provided to the plurality of users.
However, Lanier disclosed based at least in part on the received depth data, generating a point cloud (col. 12, lines 60-63, note that the stream of depth data can be derived from IR sensors (e.g., time of flight, etc.) and can be represented as a point cloud reflective of the real scene. The point cloud can represent a set of data points or depth pixels associated with surfaces of real objects and/or the real scene configured in a three-dimensional coordinate system (e.g., worldspace).); and based at least in part on the point cloud, generating a plurality of perspectives from which the 360-degree video is provided to the plurality of users (col. 12, lines 1-4, note that the depth pixels can be mapped into a grid. The grid of depth pixels can indicate a distance between real objects in the real scene and the cameras and/or sensors. The grid of depth pixels that correspond to the volume of space that is observable from the cameras and/or sensors can be called a depth space. The depth space can be utilized by the rendering module 132 (in the devices 108) for determining how to render virtual content in the mixed reality display).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach based at least in part on the received depth data, generating a point cloud; and based at least in part on the point cloud, generating a plurality of perspectives from which the 360-degree video is provided to the plurality of users. The suggestion/motivation for doing so would have been in order to provide enhance mixed reality social interactions between users in mixed reality environments (col. 3, lines 2-4). Therefore, it would have been obvious to combine Valdivia with Lanier to obtain the invention as specified in claim 12.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hamrick et al. (US Patent Number 9,223,786 B1) disclosed communication in a sensory immersive motion capture simulation environment.
Shuster et al. (US Publication Number 2016/0071323 A1) disclosed a method of displaying images on an immersive display. The method includes receiving information from an external sensor or input device of the immersive display, based on the information received, detecting an object that conflicts with a virtual reality space, adjusting at least one dimension of virtual reality space to provide an adjusted virtual reality for display on the immersive display to accommodate for the object, and displaying the adjusted virtual reality on the display of the immersive display.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Hilina K Demeter whose telephone number is (571) 270-1676.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon could be reached at (571) 270- 0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about PAIR system, see http://pari-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HILINA K DEMETER/ Primary Examiner, Art Unit 2617
Prosecution Insights