DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 10-14, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Jayaram (US 20220292784 A1) in view of Fulks (US 20140188990 A1).
As per claim 1, Jayaram teaches the claimed:
1. A method, comprising:
receiving image data and corresponding image metadata from each of a plurality of mobile devices, (Jayaram [0038]: “From the mobile device 321, the registration server 311 receives images and corresponding image metadata.”) each of the mobile devices configured to display a view of a venue for an event and to display graphics over the view of the venue, (Jayaram [0050]: “a mobile device 321 that can be used for displaying graphics of a view at a venue … in which the view of the venue is presented on a display”) and each of the mobile devices independently maintaining a corresponding internal coordinate system (Jayaram [0103]: “independently perform the registration process for each of the mobile devices, providing them with their corresponding transformation between the mobile device's coordinate system and the real world coordinate system”.) and the corresponding image metadata including information specifying a location and an orientation of the corresponding mobile device within the venue for the event in the internal coordinate system of the corresponding mobile device when capturing the image data; (Jayaram [0051]: “The metadata from the sensors 507 can provide information on the pose (location and orientation) of the camera 501 when capturing the image data, but will be within the mobile device's internal coordinate system that may only loosely be aligned with the real world coordinate system.”)
generating for each of the mobile devices a corresponding coordinate transformation between the mobile device's independently maintained internal coordinate system and a real world coordinate system for the venue from the image data and corresponding image metadata from each of the mobile devices (Jayaram [0103]: “The registration server 311 can concurrently and independently perform the registration process for each of the mobile devices, providing them with their corresponding transformation between the mobile device's coordinate system and the real world coordinate system and with their own set of tracking points and reference images.”) and from a model of the venue in the real world coordinate system for the venue; (Jayaram [0121]: “The method further includes building of a model of the venue in the real world coordinate system from the point features of the venue and the locations of the set of fiducials features in the real world coordinate system and generating for each of the mobile devices a corresponding transformation between the mobile device's coordinate system and the real world coordinate system from the image data and image metadata from each of the mobile devices and the model of the venue in the real world coordinate system.”)
transmitting to each of the mobile devices the corresponding coordinate transformation between the mobile device's independently maintained internal coordinate system and the real world coordinate system for the venue; (Jayaram [0121]: “retrieve point features of a venue for an event in a first coordinate system and locations of a set of fiducials features for the venue in a real world coordinate system … transmit to each of the mobile devices the corresponding transformation between the mobile device's coordinate system and the real world coordinate system.”)
Jayaram alone does not explicitly teach the remaining claim limitations.
However, Jayaram in combination with Fulks teaches the claimed:
receiving, from a first of the mobile devices, a request to join a user group comprising one or more second ones of the mobile devices; and (Fulks [0004]: “a formation of at least one user group network session by a first device. … determine a request from at least one second device for an access to the at least one user group”)
transmitting to the first mobile device graphics to be displayed over the view of the venue indicating a location in the real world coordinate system of the second mobile devices within the view of the venue.
(Jayaram [0093]: “At step 1311, the mobile device 321 aligns its coordinate system with the real world coordinate system based on the transformation between the mobile device's coordinate system and the real world coordinate system. This can include retrieving, for each frame of the images, tracking position and orientation, converting these to real world coordinates, and drawing 3D graphics content from the content server over the images.”
Jayaram [0096]: “the inputs (image data and image metadata from the mobile devices 321 and point features, large scale features, and shape features from the feature database 309) can be received through the network interfaces 450 and the outputs (the coordinate transformations and tracking points and template images) transmitted to the mobile device 321 by the network interfaces 450.”
Jayaram [0120]: “receiving from each of a plurality of mobile devices image data and image metadata, the image metadata including information on the mobile device's location and orientation within a venue for an event in the mobile device's coordinate system and retrieving, from one or more databases, point features of the venue in a first coordinate system and locations of a set of fiducials features for the venue in a real world coordinate system.”
Jayaram teaches the input, which include the image data and metadata, that are transmitted to the mobile devices as coordinates of the location of each device at a venue, which the mobile device can view on its display as a graphic over the images.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the user group as taught by Fulks with the system of Jayaram in order to communicate, identify, and interact with other users in a virtual or augmented reality.
As per claim 10, this claim is similar in scope to limitations recited in claim 1, and thus is rejected under the same rationale.
As per claim 2, Jayaram and Fulks teaches the claimed:
2. The method of claim 1, wherein receiving the request to join the user group from the first mobile device includes a request to form the user group, the method further comprising:
forming a data base for the user group including an identifier and location for each member of the user group.
(Fulks [0087]: “The data structure of the group may comprise but is not restricted to a set of identifiers namely group, place (geo-location Long/Lat), and device type.”
Fulks [0032]: “allow a user which is a member of the group to broadcast and update to other members of the group relative positions”
Fulks teaches identifiers for each device type, and since each user will have their own device, thus there is an identifier for each of the users in the group.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the user group as taught by Fulks with the system of Jayaram in order to communicate, identify, and interact with other users in a virtual or augmented reality.
As per claim 11, this claim is similar in scope to limitations recited in claim 2, and thus is rejected under the same rationale.
As per claim 3, Jayaram and Fulks teaches the claimed:
3. The method of claim 2, further comprising:
periodically updating the location for each of the members of the user group.
(Fulks [0032]: “allow a user which is a member of the group to broadcast and update to other members of the group relative positions”
Fulks [0077]: “a monitoring of the location information continuously, periodically, according to a schedule, on demand, or a combination thereof. … causes an update of the location information for one or more mobile devices based, at least in part, on the monitoring.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the user group as taught by Fulks with the system of Jayaram in order to communicate, identify, and interact with other users in a virtual or augmented reality.
As per claim 12, this claim is similar in scope to limitations recited in claim 3, and thus is rejected under the same rationale.
As per claim 4, Jayaram and Fulks teaches the claimed:
4. The method of claim 1, further comprising:
sending an invitation to the first mobile device to join the user group, wherein the request to join the user group from the first mobile device is in response to the invitation.
(Fulks [0071]: “the network management platform 111 determines a request from at least one second device for an access to the at least one user group network session.”
Fulks [0075]: “the network management platform 111 causes, at least in part, a granting of the access to the at least one user group based, at least in part, on an invitation from the first device to the at least one second device.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the user group as taught by Fulks with the system of Jayaram in order to communicate, identify, and interact with other users in a virtual or augmented reality.
As per claim 13, this claim is similar in scope to limitations recited in claim 4, and thus is rejected under the same rationale.
As per claim 5, Jayaram and Fulks teaches the claimed:
5. The method of claim 4, further comprising:
receiving a request from one of the second mobile devices to invite the first mobile device to join the group, wherein the invitation is sent to the first mobile device in response to the request from the one of the second mobile devices to invite the first mobile device.
(Fulks [0075]: “In this regard, the first user may provide for the generation of an invitation to the second device. As such, in response to the generation of this invitation, user associated with the second device may accept the request to access the user group network session.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the user group as taught by Fulks with the system of Jayaram in order to communicate, identify, and interact with other users in a virtual or augmented reality.
As per claim 14, this claim is similar in scope to limitations recited in claim 5, and thus is rejected under the same rationale.
As per claim 19, the reasons and rationale for the rejection of claim 1 and 8 are incorporated herein. In particular, only additional features unique to claim 19 that were not present in claim 1 or 8 will be explicitly addressed here.
As per claim 19, Jayaram teaches the claimed: (strikethrough taught in previous claims)
19. A method, comprising:
(Jayaram [0056]: “mobile devices 321 are registered with a server system including a registration server 311 at step 607. This is done by each mobile device 321 sending the registration server 311 image data and metadata, that will be in the coordinate system of the mobile device, to the registration server.”
Jayaram [0122]: “transmitting, from a mobile device to a server system, image data of a venue and image metadata, the image metadata including location information and orientation for the image data in the mobile device's coordinate system”)
(Jayaram [0056]: “For each mobile device 321, the registration server can then build a transformation for converting positions/locations between the mobile device's coordinate system to a real world coordinate system. The registration server 311 also sends each mobile device 321 template images with a set of tracking points within each of the template images at step 609. The template images with tracking points allow for each of the mobile devices 321 to maintain an accurate transformation between the mobile device's coordinate system and the real world coordinate system as the mobile device changes its pose (i.e., location and orientation).”
Jayaram [0122]: “in response to transmitting the image data and image metadata, receiving at the mobile device a transformation between the mobile device's coordinate system and the real world coordinate system from the server system”
Jayaram [0129]: “generating by the server system of a corresponding transformation between each of the mobile device's coordinate system and the real world coordinate system from the corresponding image data and image metadata of each of the mobile devices and the model of the venue in the real world coordinate system built from the image data and image metadata from the plurality of mobile devices”)
transmitting, from the mobile device to the server system,
(Jayaram [0056]: “mobile devices 321 are registered with a server system including a registration server 311 at step 607.”)
(Jayaram [0122]: “from the mobile device to the server system, requests for graphics to be displayed by the mobile device”.)
(Jayaram [0127]: “convert the graphics into a coordinate system of the mobile device based on the transformation between the mobile device's coordinate system and the real world coordinate system; and present the converted the graphics over the view of the venue.”)
Claims 6-8, 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Jayaram (US 20220292784 A1) in view of Fulks (US 20140188990 A1) in further view of Moll (US 20240139611 A1).
As per claim 6, Jayaram and Fulks alone does not explicitly teach the claimed limitations.
However, Jayaram and Fulks in combination with Moll teaches the claimed:
6. The method of claim 1, wherein the graphics indicating a location for members of the group include symbol.
(Moll [0049]: “The map system 210 provides various geographic location functions and supports the presentation of map-based media content and messages by the messaging client … the map system 210 enables the display of user icons or avatars (e.g., stored in profile data 316) on a map to indicate a current or past location of “friends” of a user”.)
Moll teaches the map system which presents media content with location function that display icons or avatars of user friends, which correspond to the graphics that indicate a location of the group that includes symbols.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the user icon or avatar as taught by Moll with the system of Jayaram as modified by Fulks in order to create a personally identifiable image to a user or member that is easily identifiable through symbols.
As per claim 15, this claim is similar in scope to limitations recited in claim 6, and thus is rejected under the same rationale.
As per claim 7, Jayaram and Fulks alone does not explicitly teach the claimed limitations.
However, Jayaram and Fulks in combination with Moll teaches the claimed:
7. The method of claim 1, wherein the graphics indicating a location for each of the members of the group include user specific text.
(Moll [0049]: “The map system 210 provides various geographic location functions and supports the presentation of map-based media content and messages by the messaging client … as well as media content (e.g., collections of messages including photographs and videos) generated by such friends, within the context of a map.”
Moll teaches the map system which presents media content with location function that also supports messages or text generated by the user friends, which correspond to the user specified text.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the user messages as taught by Moll with the system of Jayaram as modified by Fulks in order to enhance communication, engagement, and clarity with other users.
As per claim 16, this claim is similar in scope to limitations recited in claim 7, and thus is rejected under the same rationale.
As per claim 8, Jayaram and Fulks alone does not explicitly teach the claimed limitations.
However, Jayaram and Fulks in combination with Moll teaches the claimed:
8. The method of claim 1, further comprising:
receiving from a group member a request to display specified graphics as part of graphics indicating the requesting group members location; and
(Fulks [0082]: “this network can be enhanced to support messaging between group members to mark specific places, choose rendezvous destinations, broadcast places of interest and create a virtual local space for users to interact in a variety of ways.”
Moll [0049]: “A user can furthermore share his or her location and status information (e.g., using an appropriate status avatar) with other users of the messaging system 100 via the messaging client 104, with this location and status information being similarly displayed within the context of a map interface of the messaging client 104 to selected users.”
Fulks teaches the group members that can mark and broadcast specific locations, where users can interact in a virtual space, which is the graphics being displayed, and additionally, Moll teaches the user being able to share their location with others, where the user will also be able to see other’s locations on the map interface.)
including the specified graphics as part of graphics transmitted to the first mobile device graphics to be displayed over the view of the venue indicating the location of the requesting group member.
(Fulks [0082]: “causes, at least in part, a marking of one or more places, a broadcasting of the one or more places, or a combination thereof, by the first device, the at least one second device, or a combination thereof. … wherein the recipient can look at the camera and look across the venue and find the rendezvous point.”
Fulks teaches the marking and broadcasting places that are displayed in the AR view to locate other group members from the first device.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the network as taught by Fulks with the system of Jayaram in order to allow for more flexibility in what graphics the user wants on their own device Also to use the sharing of location of other users as taught by Moll with the system of Jayaram in order to improve real time coordination, spatial accuracy, and interactions with other users in an augmented reality.
As per claim 17, this claim is similar in scope to limitations recited in claim 8, and thus is rejected under the same rationale.
As per claim 20, Jayaram and Fulks alone does not explicitly teach the claimed limitations.
However, Jayaram and Fulks in combination with Moll teaches the claimed:
20. The method of claim 19, further comprising: selecting which of the other members of the user group whose locations are to be displayed over the view of the venue.
(Fulks [0082]: “this network can be enhanced to support messaging between group members to mark specific places, choose rendezvous destinations, broadcast places of interest and create a virtual local space for users to interact in a variety of ways.”
Moll [0049]: “the map system 210 enables the display of user icons or avatars (e.g., stored in profile data 316) on a map to indicate a current or past location of “friends” of a user, as well as media content (e.g., collections of messages including photographs and videos) generated by such friends, within the context of a map. … A user can furthermore share his or her location and status information (e.g., using an appropriate status avatar) with other users of the messaging system 100 via the messaging client 104, with this location and status information being similarly displayed within the context of a map interface of the messaging client 104 to selected users.”
Fulks teaches the group members that can mark and broadcast specific locations, where users can interact in a virtual space, which is the graphics being displayed, and additionally, Moll teaches the user being able to share their location with others and to indicate past or current locations of friends, which can correspond to the selecting which other members’ location to view and where the user will also be able to see other’s locations on the map interface, which is the venue.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the network as taught by Fulks with the system of Jayaram in order to allow for more flexibility in what graphics the user wants on their own device Also to use the sharing of location of other users as taught by Moll with the system of Jayaram in order to improve real time coordination, spatial accuracy, and interactions with other users in an augmented reality.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Jayaram (US 20220292784 A1) in view of Fulks (US 20140188990 A1) in further view of Mashiach (US 20230129399 A1).
As per claim 9, Jayaram and Fulks alone does not explicitly teach the claimed limitations.
However, Jayaram and Fulks in combination with Mashiach teaches the claimed:
9. The method of claim 1, further comprising:
receiving from a group member a request for graphics to be displayed over the view of the venue indicating a location of a vendor at the venue; and
(Mashiach [0041]: “The system may allow members to view the member directory. Members may also selectively view profiles in the member directory. Planners, for example, may filter the venues and/or vendors they wish to view or otherwise interact with, based on criteria that they determine. Such criteria may include, for example, geographic location(s)”
Jayaram [0050]: “the view of the venue is presented on a display 503, such as a screen with the graphics content also represented on the display.”
Mashiach teaches the system that allows members to request and filter specific criteria of which venues or vendors they want to look at, and Jayaram can present the venue with the vendor locations on the display in the graphical content.)
transmitting to the first mobile device graphics to be displayed over the view of the venue indicating a location in the real world coordinate system of the vendor within the view of the venue.
(Jayaram [0093]: “At step 1311, the mobile device 321 aligns its coordinate system with the real world coordinate system based on the transformation between the mobile device's coordinate system and the real world coordinate system. This can include retrieving, for each frame of the images, tracking position and orientation, converting these to real world coordinates, and drawing 3D graphics content from the content server over the images.”
Jayaram [0096]: “the inputs (image data and image metadata from the mobile devices 321 and point features, large scale features, and shape features from the feature database 309) can be received through the network interfaces 450 and the outputs (the coordinate transformations and tracking points and template images) transmitted to the mobile device 321 by the network interfaces 450.”
Jayaram [0043]: “The pixel location of fiducial markers can be identified in a variety of the survey images and their 3D coordinates determined via triangulation using the camera parameters”
Jayaram [0062]: “The position of the fiducials are determined so that their points' coordinates in the real world coordinate system is well known. This can be done by placing the fiduciaries at locations with well-known coordinates, such as is often the case for features in the venue (e.g., sprinkler locations of a golf course), by accurately measuring the locations of fiduciaries by a GPS or other positioning device, or a combination of these.”
Jayaram teaches the fiducial points of the venue, where the positions of these points are in a real world coordinate system, in which these fiducial points can indicate well known vendors within the venue, as the example describes sprinklers on a gold course, vendors in a venue can be indicated in a similar way. This information can be inputted and received through network interfaces to provide the outputs transmitted to the mobile device.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the filtering and requesting of vendor or venues as taught by Mashiach with the system of Jayaram as modified by Fulks in order to improve navigation and instructiveness with other users in the augmented reality.
As per claim 18, this claim is similar in scope to limitations recited in claim 1, and thus is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA SUO whose telephone number is (571) 272-8387. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached on (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSHUA SUO/Examiner, Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616
Prosecution Insights