DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/4/25 has been entered.
Summary
This action is a responsive to the request for continued examination filed on 9/4/2025.
Claim 13 has been canceled.
Claim 22 is new.
Claims 1-12, 14-22 are pending and have been examined.
Claims 1-12, 14-22 are rejected.
Response to Arguments
Rejection of Claims under 35 USC 103
Applicant’s Response:
Applicant submits that the cited references fail to teach the newly added limitation of:
“send a description representing the subset of information to a second device”.
Examiner’s Response:
Applicant’s arguments with respect to claims 1 and 16 have been considered but are moot because the arguments are directed to amended subject matter properly addressed with the newly cited reference of Petrov et al. (US 20230252736 A1).
The combination of Kundu et al. (US 20230298226 A1) and Campbell et al. (US 20220255763 A1) and Petrov et al. (US 20230252736 A1) teaches the language of independent claims 1 and 16.
The combination of Kundu et al. (US 20230298226 A1) and Spivackv et al. (US 20190107991 A1) and Petrov et al. (US 20230252736 A1) teaches the language of independent claim 8.
All remaining arguments are now moot in regards to the new rejection.
Claim Objections
Claims 8 and 16 objected to because of the following informalities:
Claim 8 – Claims filed 3/17/25 include the limitation of “in response to receiving the subset of sensor information to a second device, send the content to the first device for display on a display of the first device in proximity to the feature identified from the description”. It is missing sections from the claims filed 9/4/25. For examination, this limitation will be treated as part of the claims filed 9/4/25.
Claim 16 – Claims filed 3/17/25 include the limitation of “and displaying content associated with the second device in a field of view of a real-world environment corresponding to the subset of sensor information”. It is missing from the claims filed 9/4/25. For examination, this limitation will be treated as part of the claims filed 9/4/25.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-7, 16-17 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Kundu et al. (US 20230298226 A1) and further in view of Campbell et al. (US 20220255763 A1) and Petrov et al. (US 20230252736 A1).
As to claim 1, Kundu et al. teaches a computing device comprising: a processor; a memory configured with instructions to (See ¶ [0220], Teaches that Computing platform(s) 1502 may include electronic storage 1528, one or more processors 1530, and/or other components):
receive information from the sensor (See ¶¶ [0085], [0091], Teaches that , at step 802 one or more of the systems described herein may receive media content. The systems described herein may perform step 802 in a variety of ways. In one example, the media content may be received and stored locally. A user may capture media using a computing device 901, which may correspond to a user device such as a mobile device, laptop, etc. and/or may correspond to an artificial reality device, such as augmented reality glasses, augmented reality watch, etc. );
filter the sensor based on the criteria to generate a subset of information (See ¶¶ [0092]-[0093] Teaches that detection modules 905 may be run on the segmented data to determine whether they cross a privacy score threshold. For example, detection modules 905 may use features such as where faces are looking (directly at the camera or not), media metadata such as a location which can map to the user's house, etc. In addition, the privacy score may be calculated and analyzed for each segmented element. Once the privacy score value crosses a privacy threshold, the media may be fixed. However, detection modules 905 may also check privacy settings 907 to determine whether a particular element should remain unmodified. This process may occur locally with a companion app, or remotely on a server. The user may be required to give consent for the server processing in order to protect other people's privacy. GAN models 904 may be used to generate synthetic data to replace the privacy invasive portions of the media. Thus, a synthetic face may be displayed, rather than a blur or a black box. This process may also be applied to other forms of media, such as videos, audio (e.g., voice), etc.).
However, it does not expressly teach the details of receive a criteria for filtering, the criteria relating to a sensor recording a real- world environment around the computing device; and send a description representing the subset of information to a second device; and a display configured to display content associated with the second device proximate to a feature included in the description in a field of view including the real-world environment around the computing device.
Campbell et al., from analogous art, teaches receive a criteria for filtering, the criteria relating to a sensor recording a real- world environment around the computing device (See ¶¶ [0083], [0077], [0155], Teaches that a user device 102 receives privacy settings associated with a user. The user device 102 can encrypt the new or updated privacy settings. The privacy settings can include new settings or updates to previously received settings. A user associated with a user device 102 can also provide user privacy settings, or privacy settings. The user has control over the user's own user data, or environment data, generated by their device(s). The user also has control over what user data is accessible from other sources (e.g., (e.g., third parties, other devices, the user herself, or any other data available). The data may include data (a) captured from sensors (which may be, e.g., operatively coupled to the frame 1012 or otherwise attached to the user 1004), such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, or gyros;).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Campbell et al. into Kundu et al. in order to permit the user to control the protection level(s) for this information, so that the user is in control of the data collection and/or sharing of user data based on the user's personal privacy preferences (See Campbell et al. ¶ [0022]).
However, it does not expressly teach the details of send a description representing the subset of information to a second device; and a display configured to display content associated with the second device proximate to a feature included in the description in a field of view including the real-world environment around the computing device.
Petrov et al., from analogous art, teaches send a description representing the subset of information to a second device (See ¶ [0126], [0054]-[0055], Teaches that the method 700 includes obtaining (e.g., receiving, retrieving, or determining/generating) a physical environment descriptor associated with the current FOV of the physical environment. In some implementations, as represented by block 742, the physical environment descriptor includes at least one of object recognition information, instance segmentation information, semantic segmentation information, SLAM information, or the like associated with the current FOV of the physical environment.);
and a display configured to display content associated with the second device proximate to a feature included in the description in a field of view including the real-world environment around the computing device (See ¶¶ [0128], [0054]-[0055], Teaches that the method 700 includes determining whether the current FOV of the physical environment includes the real-world object based on the physical environment descriptor. In accordance with a determination that the current FOV of the physical environment includes the real-world object, the method 700 continues to block 760. In accordance with a determination that the current FOV of the physical environment does not include the real-world object, the method 700 continues to block 730 (e.g., the computing system continues obtaining images(s) associated with the current FOV of the physical environment. As one example, with reference to FIGS. 2 and 4C, in response to determining that the electronic message is attached to or associated with the real-world object, the computing system or a component thereof (e.g., the surfacer engine 439) is configured to determine whether a current FOV of the physical environment 105 includes the real-world object.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Petrov et al. into the combination of Kundu et al and Campbell et al. in order to obtain (e.g., receive, retrieve, or determine/generate) an environment descriptor based on the input data and update the environment descriptor over time (See Petrov et al. ¶ [0043]).
As to claim 3, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 1 above. Kundu et al. further teaches wherein the sensor includes a global positioning system (See ¶ [0300], Teaches that These sensors may include cameras, IR sensors, heat sensors, motion sensors, GPS receivers, or in some cases, sensors that detect a user's eye movements),
the criteria includes an exclusion zone, and filtering the information based on the criteria further comprises filtering the information to remove data within the exclusion zone to generate the subset of information (See ¶ [0235], Teaches that privacy settings may allow a user to specify one or more geographic locations from which objects can be accessed. Access or denial of access to the objects may depend on the geographic location of a user who is attempting to access the objects. As an example, and not by way of limitation, a user may share an object and specify that only users in the same city may access or view the object. As another example and not by way of limitation, a first user may share an object and specify that the object is visible to second users only while the first user is in a particular location. If the first user leaves the particular location, the object may no longer be visible to the second users. As another example and not by way of limitation, a first user may specify that an object is visible only to second users within a threshold distance from the first user. If the first user subsequently changes location, the original second users with access to the object may lose access, while a new group of second users may gain access as they come within the threshold distance of the first user.).
As to claim 4, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 1 above. Kundu et al. further teaches wherein the sensor includes a global positioning system (See ¶ [0300], Teaches that These sensors may include cameras, IR sensors, heat sensors, motion sensors, GPS receivers, or in some cases, sensors that detect a user's eye movements),
the criteria includes an inclusion zone, and filtering the information based on the criteria further comprises filtering the information to remove data outside the inclusion zone to generate the subset of information (See ¶ [0235], Teaches that privacy settings may allow a user to specify one or more geographic locations from which objects can be accessed. Access or denial of access to the objects may depend on the geographic location of a user who is attempting to access the objects. As an example, and not by way of limitation, a user may share an object and specify that only users in the same city may access or view the object. As another example and not by way of limitation, a first user may share an object and specify that the object is visible to second users only while the first user is in a particular location. If the first user leaves the particular location, the object may no longer be visible to the second users. As another example and not by way of limitation, a first user may specify that an object is visible only to second users within a threshold distance from the first user. If the first user subsequently changes location, the original second users with access to the object may lose access, while a new group of second users may gain access as they come within the threshold distance of the first user.).
As to claim 5, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 1 above. Kundu et al. further teaches wherein the sensor includes at least one of a lidar or a camera (See ¶¶ [0290], [0071], Teaches that augmented-reality system 1800 and/or virtual-reality system 1900 may include one or more optical sensors, such as two-dimensional (2D) or 3D cameras, structured light transmitters and detectors, time-of-flight depth sensors, single-beam or sweeping laser rangefinders, 3D LiDAR sensors, and/or any other suitable type or form of optical sensor. Image frames may be captured by the camera of a device at fixed intervals (e.g., every millisecond, every ten milliseconds, every second, etc.) and/or in response to user input (e.g., pressing and/or holding a “record” or “capture” button).),
the criteria includes at least one exclusion feature, and filtering the information based on the criteria further comprises filtering the information to remove data related to the at least one exclusion feature to generate the subset of information (See ¶¶ [0143], [0144], Teaches that the disclosed systems and methods may enable an owner or user of an AR system to share a live preview of captured data with bystanders. In some embodiments, the disclosed systems and methods may send the live preview from the AR system to a companion application running on another device (e.g., a smartphone or smartwatch) so that the bystanders can see how privacy is being protected prior to capture or recording. The disclosed systems and methods may enable bystanders to hold a visual pattern (e.g., a QR code) in front of an AR system's camera to filter recently captured photos/videos and/or to blur faces that match with the bystander or the bystanders' companions.).
As to claim 6, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 1 above. Kundu et al. further teaches wherein the subset of information includes at least one of: an object, a surface, a context, a weather type, an event, a person, or a location (See ¶¶ [0087], [0145] Teaches that a privacy portion of a media content may correspond to faces, certain text (e.g., addresses, license plate numbers, etc.) recognizable objects, private locations (e.g., a person's bedroom, office, etc.) that are captured in the media content. The systems described herein may perform step 806 in a variety of ways. In one example, detecting the privacy portion may be based on a privacy score applied to each segmented element. For instance, the privacy score may be based on at least one of a face orientation, a media metadata, a location, or user settings. The user settings may include user selected persons or objects that are not considered private. The disclosed systems may blur faces in captured images or videos unless the person who is in the captured images or video has given permission to the system not to do so. For a person that wants to be captured by a specific AR system (or a category of these capturing systems), the disclosed systems may enable the person to grant permission through some kind of interface (e.g., using a face-identification system). If permissions are tied to a particular face identity (or something similar with encrypted information about the person's facial features), then the disclosed systems may guarantee that the identity of the one in the capture is the same as the one who grants permission.).
As to claim 7, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 1 above. Kundu et al. further teaches wherein the sensor comprises any combination of: a camera, a microphone, an inertial measurement unit, a global positioning system, or a lidar (See ¶¶ [0290], [0071], [0300], Teaches that augmented-reality system 1800 and/or virtual-reality system 1900 may include one or more optical sensors, such as two-dimensional (2D) or 3D cameras, structured light transmitters and detectors, time-of-flight depth sensors, single-beam or sweeping laser rangefinders, 3D LiDAR sensors, and/or any other suitable type or form of optical sensor. Image frames may be captured by the camera of a device at fixed intervals (e.g., every millisecond, every ten milliseconds, every second, etc.) and/or in response to user input (e.g., pressing and/or holding a “record” or “capture” button). These sensors may include cameras, IR sensors, heat sensors, motion sensors, GPS receivers, or in some cases, sensors that detect a user's eye movements).
As to claim 16, Kundu et al. teaches a method, comprising: receiving information from a sensor recording a real-world environment around the first device (See ¶¶ [0085], [0091], Teaches that , at step 802 one or more of the systems described herein may receive media content. The systems described herein may perform step 802 in a variety of ways. In one example, the media content may be received and stored locally. A user may capture media using a computing device 901, which may correspond to a user device such as a mobile device, laptop, etc. and/or may correspond to an artificial reality device, such as augmented reality glasses, augmented reality watch, etc. ).
However, it does not expressly teach the details of receiving a criteria for filtering from a first device; and sending a description representing a subset of information to a second device based on the information and the criteria; and displaying content associated with the second device in a field of view of a real-world environment corresponding to the subset of sensor information.
Campbell et al., from analogous art, teaches receiving a criteria for filtering from a first device (See ¶¶ [0083], [0077], [0155], Teaches that a user device 102 receives privacy settings associated with a user. The user device 102 can encrypt the new or updated privacy settings. The privacy settings can include new settings or updates to previously received settings. A user associated with a user device 102 can also provide user privacy settings, or privacy settings. The user has control over the user's own user data, or environment data, generated by their device(s). The user also has control over what user data is accessible from other sources (e.g., (e.g., third parties, other devices, the user herself, or any other data available). The data may include data (a) captured from sensors (which may be, e.g., operatively coupled to the frame 1012 or otherwise attached to the user 1004), such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radio devices, or gyros;).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Campbell et al. into Kundu et al. in order to permit the user to control the protection level(s) for this information, so that the user is in control of the data collection and/or sharing of user data based on the user's personal privacy preferences (See Campbell et al. ¶ [0022]).
However, it does not expressly teach the details of sending a description representing a subset of information to a second device based on the information and the criteria; and displaying content associated with the second device in a field of view of a real-world environment corresponding to the subset of sensor information.
Petrov et al., from analogous art, teaches of sending a description representing a subset of information to a second device based on the information and the criteria (See ¶ [0126], [0054]-[0055], Teaches that the method 700 includes obtaining (e.g., receiving, retrieving, or determining/generating) a physical environment descriptor associated with the current FOV of the physical environment. In some implementations, as represented by block 742, the physical environment descriptor includes at least one of object recognition information, instance segmentation information, semantic segmentation information, SLAM information, or the like associated with the current FOV of the physical environment.);
and displaying content associated with the second device in a field of view of a real-world environment corresponding to the subset of sensor information (See ¶¶ [0128], [0054]-[0055], Teaches that the method 700 includes determining whether the current FOV of the physical environment includes the real-world object based on the physical environment descriptor. In accordance with a determination that the current FOV of the physical environment includes the real-world object, the method 700 continues to block 760. In accordance with a determination that the current FOV of the physical environment does not include the real-world object, the method 700 continues to block 730 (e.g., the computing system continues obtaining images(s) associated with the current FOV of the physical environment. As one example, with reference to FIGS. 2 and 4C, in response to determining that the electronic message is attached to or associated with the real-world object, the computing system or a component thereof (e.g., the surfacer engine 439) is configured to determine whether a current FOV of the physical environment 105 includes the real-world object.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Petrov et al. into the combination of Kundu et al and Campbell et al. in order to obtain (e.g., receive, retrieve, or determine/generate) an environment descriptor based on the input data and update the environment descriptor over time (See Petrov et al. ¶ [0043]).
As to claim 17, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 16 above. However, it does not expressly teach the details of wherein sending the description representing the subset of information to the second device further comprises: generating a rendering of static components the subset of information.
Petrov et al., from analogous art, teaches of wherein sending the description representing the subset of information to the second device further comprises: generating a rendering of static components the subset of information (See ¶¶ [0128], [0054]-[0055], Teaches that the method 700 includes determining whether the current FOV of the physical environment includes the real-world object based on the physical environment descriptor. In accordance with a determination that the current FOV of the physical environment includes the real-world object, the method 700 continues to block 760. In accordance with a determination that the current FOV of the physical environment does not include the real-world object, the method 700 continues to block 730 (e.g., the computing system continues obtaining images(s) associated with the current FOV of the physical environment. As one example, with reference to FIGS. 2 and 4C, in response to determining that the electronic message is attached to or associated with the real-world object, the computing system or a component thereof (e.g., the surfacer engine 439) is configured to determine whether a current FOV of the physical environment 105 includes the real-world object.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Petrov et al. into the combination of Kundu et al and Campbell et al. and Petrov et al. in order to obtain (e.g., receive, retrieve, or determine/generate) an environment descriptor based on the input data and update the environment descriptor over time (See Petrov et al. ¶ [0043]).
As to claim 21, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the method according to claim 16 above. Kundu et al. further teaches wherein the subset of information includes at least one of: an object, a surface, a context, a weather type, an event, a person, or a location (See ¶¶ [0087], [0145] Teaches that a privacy portion of a media content may correspond to faces, certain text (e.g., addresses, license plate numbers, etc.) recognizable objects, private locations (e.g., a person's bedroom, office, etc.) that are captured in the media content. The systems described herein may perform step 806 in a variety of ways. In one example, detecting the privacy portion may be based on a privacy score applied to each segmented element. For instance, the privacy score may be based on at least one of a face orientation, a media metadata, a location, or user settings. The user settings may include user selected persons or objects that are not considered private. The disclosed systems may blur faces in captured images or videos unless the person who is in the captured images or video has given permission to the system not to do so. For a person that wants to be captured by a specific AR system (or a category of these capturing systems), the disclosed systems may enable the person to grant permission through some kind of interface (e.g., using a face-identification system). If permissions are tied to a particular face identity (or something similar with encrypted information about the person's facial features), then the disclosed systems may guarantee that the identity of the one in the capture is the same as the one who grants permission.).
Claims 2 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kundu et al. (US 20230298226 A1) and Campbell et al. (US 20220255763 A1 and Petrov et al. (US 20230252736 A1) and further in view of TICHENOR et al. (US 20230162453 A1).
As to claim 2, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 1 above. However, it does not expressly teach the details of wherein the memory is further configured with instructions to: receive the content from the second device; receive the feature selected from the description from the second device based on the description; and display the content in a field of view proximate to the feature.
TICHENOR et al., from analogous art, teaches wherein the memory is further configured with instructions to: receive the content from the second device (See ¶ [0088], Teaches that can receive a request, associated with a manifest, for a new augment. As discussed above in relation to FIG. 5 , the manifest can specify properties and/or logic for the new augment, such as display modes (e.g., one or more context factor values that must occur for the display mode to be enabled, types of surfaces on which the augment can be placed in that display mode, a location, shape, or orientation, whether the augment is moveable, a volume for audio presentation data, etc.), a title, an ID, and augment type, an owner of the augment, an initial location for the augment, etc. In some implementations, default or inherited properties and/or logic can also be specified for the augment, such as having all augments include certain display modes that correspond to artificial reality system modes (e.g., audio only, minimized, interactive, etc.).);
receive the feature selected from the description from the second device based on the description (See ¶ [0107], Teaches that process 800 can receive an identification of a surface. This can be an existing surface or a surface created to accommodate a new augment. In various implementations, an artificial reality system can have created one or more surfaces. In various implementations, surfaces can be (i) synthetic (generated automatically by the artificial reality system without regard to the environment, e.g., not world-locked), (ii) semantic (detected by machine learning recognizers e.g., hands, faces, table, or other particular objects, etc.), or (iii) geometric (identified geometries in the environment e.g., floors, walls, etc.) Thus, the artificial reality system can create surfaces by defining surfaces relative to the artificial reality system position, by identifying artificial reality environment surface geometries or object types specified for creating a surface, or in response to a user defining a surface (e.g., by performing an air tap, outlining a surface with a gesture, putting a hand, controller, or other peripheral device on the surface, defining an plane in the air with her hand to be used as a surface, interacting with an object type, such as handlebars on a bike, to define a surface, etc.));
and display the content in a field of view proximate to the feature (See ¶ [0092], Teaches that process 600 can perform an augment placement procedure. In some implementations, block 610 can be performed prior to block 606, e.g., setting an initial augment placement before providing the handle to the requestor, for the requestor to update the placement. The placement procedure can include setting a default location or a location specified in the request received at block 602 (e.g., a location based on the user's focus area, a location relative to the requesting entity such as the same surface a requesting augment is attached to or surface defined for the hands or face of a user associated with the request). In some implementations, the placement procedure can include making the augment invisible until the requestor selects a location.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of TICHENOR et al. into the combination of Kundu et al and Campbell et al. and Petrov et al. in order to provide an artificial reality environment with a 2D or 3D volume, in an artificial reality environment, that can include presentation data, context, and logic (See TICHENOR et al. ¶ [0026]).
As to claim 19, the combination of Kundu et al. and Campbell et al. and Petrov et al. teaches the method according to claim 17 above. However, it does not expressly teach the details of wherein the subset of information is a list of environmental features.
TICHENOR et al., from analogous art, teaches wherein the subset of information is a list of environmental features (See ¶¶ [0107], [0119], [0088] Teaches that process 800 can receive an identification of a surface. This can be an existing surface or a surface created to accommodate a new augment. In various implementations, an artificial reality system can have created one or more surfaces. In various implementations, surfaces can be (i) synthetic (generated automatically by the artificial reality system without regard to the environment, e.g., not world-locked), (ii) semantic (detected by machine learning recognizers e.g., hands, faces, table, or other particular objects, etc.), or (iii) geometric (identified geometries in the environment e.g., floors, walls, etc.) Thus, the artificial reality system can create surfaces by defining surfaces relative to the artificial reality system position, by identifying artificial reality environment surface geometries or object types specified for creating a surface, or in response to a user defining a surface (e.g., by performing an air tap, outlining a surface with a gesture, putting a hand, controller, or other peripheral device on the surface, defining an plane in the air with her hand to be used as a surface, interacting with an object type, such as handlebars on a bike, to define a surface, etc.). FIG. 11 is a conceptual diagram continuing illustrating the example 1000 of augments in the artificial reality space where the artificial reality system identifies virtual surfaces. In example 1000, the artificial reality system identifies geometric surfaces by automatically locating flat surfaces of at least a certain size. In this manner, the artificial reality system has automatically identified surfaces 1102 and 1104. The artificial reality system also identified a surface on the floor but the user instructed (not shown) the system to divide that surface into surfaces 1110 and 1112. The user also caused (not shown) the artificial reality system to create surface 1108 by placing her hand on the surface. Existing real and virtual objects located on these surfaces are automatically added to them. ).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of TICHENOR et al. into the combination of Kundu et al and Campbell et al. and Petrov et al. in order to provide an artificial reality environment with a 2D or 3D volume, in an artificial reality environment, that can include presentation data, context, and logic (See TICHENOR et al. ¶ [0026]).
As to claim 20, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the method according to claim 16 above. However, it does not expressly teach the details of further comprising: receiving content from the second device; receiving feature from the second device selected from the description; and sending the content and the feature to a first device.
TICHENOR et al., from analogous art, teaches further comprising: receiving content from the second device (See ¶ [0088], Teaches that can receive a request, associated with a manifest, for a new augment. As discussed above in relation to FIG. 5 , the manifest can specify properties and/or logic for the new augment, such as display modes (e.g., one or more context factor values that must occur for the display mode to be enabled, types of surfaces on which the augment can be placed in that display mode, a location, shape, or orientation, whether the augment is moveable, a volume for audio presentation data, etc.), a title, an ID, and augment type, an owner of the augment, an initial location for the augment, etc. In some implementations, default or inherited properties and/or logic can also be specified for the augment, such as having all augments include certain display modes that correspond to artificial reality system modes (e.g., audio only, minimized, interactive, etc.).);
receiving feature from the second device selected from the description (See ¶ [0107], Teaches that process 800 can receive an identification of a surface. This can be an existing surface or a surface created to accommodate a new augment. In various implementations, an artificial reality system can have created one or more surfaces. In various implementations, surfaces can be (i) synthetic (generated automatically by the artificial reality system without regard to the environment, e.g., not world-locked), (ii) semantic (detected by machine learning recognizers e.g., hands, faces, table, or other particular objects, etc.), or (iii) geometric (identified geometries in the environment e.g., floors, walls, etc.) Thus, the artificial reality system can create surfaces by defining surfaces relative to the artificial reality system position, by identifying artificial reality environment surface geometries or object types specified for creating a surface, or in response to a user defining a surface (e.g., by performing an air tap, outlining a surface with a gesture, putting a hand, controller, or other peripheral device on the surface, defining an plane in the air with her hand to be used as a surface, interacting with an object type, such as handlebars on a bike, to define a surface, etc.));
and sending the content and the feature to a first device (See ¶ [0092], Teaches that process 600 can perform an augment placement procedure. In some implementations, block 610 can be performed prior to block 606, e.g., setting an initial augment placement before providing the handle to the requestor, for the requestor to update the placement. The placement procedure can include setting a default location or a location specified in the request received at block 602 (e.g., a location based on the user's focus area, a location relative to the requesting entity such as the same surface a requesting augment is attached to or surface defined for the hands or face of a user associated with the request). In some implementations, the placement procedure can include making the augment invisible until the requestor selects a location.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of TICHENOR et al. into the combination of Kundu et al and Campbell et al. and Petrov et al. in order to provide an artificial reality environment with a 2D or 3D volume, in an artificial reality environment, that can include presentation data, context, and logic (See TICHENOR et al. ¶ [0026]).
Claims 8-9, 11, 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Kundu et al. (US 20230298226 A1) and further in view of Spivackv et al. (US 20190107991 A1) and Petrov et al. (US 20230252736 A1).
As to claim 8, Kundu et al. teaches a computing device comprising: a processor; and a memory configured with instructions to (See ¶ [0220], Teaches that Computing platform(s) 1502 may include electronic storage 1528, one or more processors 1530, and/or other components):
receive a description representing a subset of information from a first device, the subset of information being based on a sensor recording a real-world environment proximate to a first device and criteria for filtering (See ¶¶ [0092]-[0093], [0143], [0144], Teaches that detection modules 905 may be run on the segmented data to determine whether they cross a privacy score threshold. For example, detection modules 905 may use features such as where faces are looking (directly at the camera or not), media metadata such as a location which can map to the user's house, etc. In addition, the privacy score may be calculated and analyzed for each segmented element. Once the privacy score value crosses a privacy threshold, the media may be fixed. However, detection modules 905 may also check privacy settings 907 to determine whether a particular element should remain unmodified. This process may occur locally with a companion app, or remotely on a server. The user may be required to give consent for the server processing in order to protect other people's privacy. GAN models 904 may be used to generate synthetic data to replace the privacy invasive portions of the media. Thus, a synthetic face may be displayed, rather than a blur or a black box. This process may also be applied to other forms of media, such as videos, audio (e.g., voice), etc. The disclosed systems and methods may enable an owner or user of an AR system to share a live preview of captured data with bystanders. In some embodiments, the disclosed systems and methods may send the live preview from the AR system to a companion application running on another device (e.g., a smartphone or smartwatch) so that the bystanders can see how privacy is being protected prior to capture or recording. The disclosed systems and methods may enable bystanders to hold a visual pattern (e.g., a QR code) in front of an AR system's camera to filter recently captured photos/videos and/or to blur faces that match with the bystander or the bystanders' companions.).
However, it does not expressly teach the details of receive a description representing a subset of information from a first device; receive content generated by a user; receive a feature selected from the description; in response to receiving the subset of sensor information to a second device, send the content to the first device for display on a display of the first device in proximity to the feature identified from the description.
Spivack et al., from analogous art, teaches receive content generated by a user (See ¶¶ [0313], [0307]-[0308], Teaches that in 602 the user can select the text icon to create or design a billboard with text. In 604, the user can specify the text, and in 606 the user can apply a design or style (shape, color) to the text and the billboard. In 608, the user can select to share the virtual billboard object with friends or to save it to a current physical location. In 610, the virtual billboard can be sent or placed at the physical location. FIG. 6B graphically depicts additional example user interfaces 620 for creating, posting and/or sharing a virtual object s having multimedia content in accordance with embodiments of the present disclosure.);
receive a feature selected from the description (See ¶¶ [0313], [0307]-[0308], Teaches that in 602 the user can select the text icon to create or design a billboard with text. In 604, the user can specify the text, and in 606 the user can apply a design or style (shape, color) to the text and the billboard. In 608, the user can select to share the virtual billboard object with friends or to save it to a current physical location. In 610, the virtual billboard can be sent or placed at the physical location. FIG. 6B graphically depicts additional example user interfaces 620 for creating, posting and/or sharing a virtual object s having multimedia content in accordance with embodiments of the present disclosure.);
and in response to receiving the subset of sensor information to a second device, send the content to the first device for display on a display of the first device in proximity to the feature identified from the description (See ¶¶ [0313]-[0314], [0416], Teaches that in 602 the user can select the text icon to create or design a billboard with text. In 604, the user can specify the text, and in 606 the user can apply a design or style (shape, color) to the text and the billboard. In 608, the user can select to share the virtual billboard object with friends or to save it to a current physical location. In 610, the virtual billboard can be sent or placed at the physical location. FIG. 6B graphically depicts additional example user interfaces 620 for creating, posting and/or sharing a virtual object s having multimedia content in accordance with embodiments of the present disclosure. FIG. 6C graphically depicts additional example user interfaces for creating a virtual object, posting a virtual object and placing a virtual object at a physical location, in accordance with embodiments of the present disclosure. The VOB that is posted can include, a 3D object, pictures, videos, text and/or link and selected in 656 or 658. A VOB can be selected in 660 and placed in a physical location in 662. Similarly, in 664 a photo can be selected and placed at the physical location in 666. In one embodiment, for close proximity (e.g., co-located user), user interaction can be achieved when 1) user can see distance in km of friends next to their friends' name on the friends list. 2) users can see friends on map (this feature could be disabled in settings). If user taps on friend icon on map, and they were close enough to them, they could activate AR mode and then search for them using a GPS arrow. Embodiments of the present disclosure include social togetherness enabled by the augmented reality environment. Social togetherness can include, for example, giving each other objects, exploring each other's space in an augmented reality world, etc.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Spivack et al. into Kundu et al. in order to facilitate interaction with a virtual billboard associated with a physical location in the real world environment (See Spivack et al. ¶ [0052]).
However, it does not expressly teach the details of receive a description representing a subset of information from a first device.
Petrov et al., from analogous art, teaches receive a description representing a subset of information from a first device (See ¶ [0126], [0054]-[0055], Teaches that the method 700 includes obtaining (e.g., receiving, retrieving, or determining/generating) a physical environment descriptor associated with the current FOV of the physical environment. In some implementations, as represented by block 742, the physical environment descriptor includes at least one of object recognition information, instance segmentation information, semantic segmentation information, SLAM information, or the like associated with the current FOV of the physical environment.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Petrov et al. into the combination of Kundu et al and Spivack et al. in order to obtain (e.g., receive, retrieve, or determine/generate) an environment descriptor based on the input data and update the environment descriptor over time (See Petrov et al. ¶ [0043]).
As to claim 9, the combination of Kundu et al. and Spivack et al. Petrov et al teaches the device according to claim 8 above. However, it does not expressly teach the details of wherein receiving the content further comprises: executing a content generation module with selectable settings operable to generate the content.
Spivack et al., from analogous art, teaches wherein receiving the content further comprises: executing a content generation module with selectable settings operable to generate the content (See ¶¶ [0313], [0307]-[0308], Teaches that in 602 the user can select the text icon to create or design a billboard with text. In 604, the user can specify the text, and in 606 the user can apply a design or style (shape, color) to the text and the billboard. In 608, the user can select to share the virtual billboard object with friends or to save it to a current physical location. In 610, the virtual billboard can be sent or placed at the physical location. FIG. 6B graphically depicts additional example user interfaces 620 for creating, posting and/or sharing a virtual object s having multimedia content in accordance with embodiments of the present disclosure.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Spivack et al. into the combination of Kundu et al. and Spivack et al. and Petrov et al. in order to facilitate interaction with a virtual billboard associated with a physical location in the real world environment (See Spivack et al. ¶ [0052]).
As to claim 11, the combination of Kundu et al and Spivack et al. and Petrov et al. teaches the device according to claim 8 above. Kundu et al. further teaches wherein receiving the description representing the subset of information from the first device further comprises: receiving a three-dimensional rendering of static components of the subset of information (See ¶¶ [0143], [0144], [0271] Teaches that the disclosed systems and methods may enable an owner or user of an AR system to share a live preview of captured data with bystanders. In some embodiments, the disclosed systems and methods may send the live preview from the AR system to a companion application running on another device (e.g., a smartphone or smartwatch) so that the bystanders can see how privacy is being protected prior to capture or recording. The disclosed systems and methods may enable bystanders to hold a visual pattern (e.g., a QR code) in front of an AR system's camera to filter recently captured photos/videos and/or to blur faces that match with the bystander or the bystanders' companions. The artificial-reality content may include video, audio, haptic feedback, or some combination thereof, any of which may be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional (3D) effect to the viewer)).
As to claim 14, the combination of Kundu et al and Spivack et al. and Petrov et al. teaches the device according to claim 8 above. Kundu et al. further teaches wherein the subset of information includes at least one of: an object, a surface, a context, a weather type, an event, a person, or a location (See ¶¶ [0087], [0145] Teaches that a privacy portion of a media content may correspond to faces, certain text (e.g., addresses, license plate numbers, etc.) recognizable objects, private locations (e.g., a person's bedroom, office, etc.) that are captured in the media content. The systems described herein may perform step 806 in a variety of ways. In one example, detecting the privacy portion may be based on a privacy score applied to each segmented element. For instance, the privacy score may be based on at least one of a face orientation, a media metadata, a location, or user settings. The user settings may include user selected persons or objects that are not considered private. The disclosed systems may blur faces in captured images or videos unless the person who is in the captured images or video has given permission to the system not to do so. For a person that wants to be captured by a specific AR system (or a category of these capturing systems), the disclosed systems may enable the person to grant permission through some kind of interface (e.g., using a face-identification system). If permissions are tied to a particular face identity (or something similar with encrypted information about the person's facial features), then the disclosed systems may guarantee that the identity of the one in the capture is the same as the one who grants permission.).
As to claim 15, the combination of Kundu et al and Spivack et al. and Petrov et al. teaches the device according to claim 8 above. Kundu et al. further teaches wherein the sensor comprises any combination of: a camera, a microphone, an inertial measurement unit, a global positioning system, or a lidar (See ¶¶ [0290], [0071], [0300], Teaches that augmented-reality system 1800 and/or virtual-reality system 1900 may include one or more optical sensors, such as two-dimensional (2D) or 3D cameras, structured light transmitters and detectors, time-of-flight depth sensors, single-beam or sweeping laser rangefinders, 3D LiDAR sensors, and/or any other suitable type or form of optical sensor. Image frames may be captured by the camera of a device at fixed intervals (e.g., every millisecond, every ten milliseconds, every second, etc.) and/or in response to user input (e.g., pressing and/or holding a “record” or “capture” button). These sensors may include cameras, IR sensors, heat sensors, motion sensors, GPS receivers, or in some cases, sensors that detect a user's eye movements).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Kundu et al. (US 20230298226 A1) and Spivackv et al. (US 20190107991 A1) and Petrov et al. (US 20230252736 A1) and further in view of TICHENOR et al. (US 20230162453 A1).
As to claim 10, the combination of Kundu et al. and Spivack et al. and Petrov et al. teaches the device according to claim 8 above. However, it does not expressly teach the details of wherein receiving the content further comprises: executing a content association module with options operable to, upon selection, select a feature from the description.
Spivack et al., from analogous art, teaches wherein receiving the content further comprises: executing a content association module with options operable to, upon selection, select a feature from the description (See ¶ [0107], Teaches that process 800 can receive an identification of a surface. This can be an existing surface or a surface created to accommodate a new augment. In various implementations, an artificial reality system can have created one or more surfaces. In various implementations, surfaces can be (i) synthetic (generated automatically by the artificial reality system without regard to the environment, e.g., not world-locked), (ii) semantic (detected by machine learning recognizers e.g., hands, faces, table, or other particular objects, etc.), or (iii) geometric (identified geometries in the environment e.g., floors, walls, etc.) Thus, the artificial reality system can create surfaces by defining surfaces relative to the artificial reality system position, by identifying artificial reality environment surface geometries or object types specified for creating a surface, or in response to a user defining a surface (e.g., by performing an air tap, outlining a surface with a gesture, putting a hand, controller, or other peripheral device on the surface, defining an plane in the air with her hand to be used as a surface, interacting with an object type, such as handlebars on a bike, to define a surface, etc.).).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of TICHENOR et al. into the combination of Kundu et al. and Spivack et al. and Petrov et al. in order to provide an artificial reality environment with a 2D or 3D volume, in an artificial reality environment, that can include presentation data, context, and logic (See TICHENOR et al. ¶ [0026]).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Kundu et al. (US 20230298226 A1) and Spivackv et al. (US 20190107991 A1) and Petrov et al. (US 20230252736 A1) and further in view of LADAVAC et al. (US 20230394767 A1).
As to claim 12, the combination of Kundu et al and Spivack et al. and Petrov et al. teaches the device according to claim 11 above. However, it does not expressly teach the details of wherein the three-dimensional rendering of the subset of information is a clay rendering.
LADAVAC et al., from analogous art, teaches wherein the three-dimensional rendering of the subset of information is a clay rendering (See ¶¶ [0030], [0059], Teaches that the mesh can be defined with sufficiently reduced complexity such that it can be processed, transmitted, and/or rendered for display quickly and stored in reduced storage space compared to other representations of surfaces and objects. For example, such a mesh is commonly used in applications such as efficient rendering of terrains, medical images, or objects modeled in a clay-like manner. Common applications include efficient rendering of terrains, medical images, or objects modeled in a clay-like manner. For example, the mesh can be used in the generation of a feature surface of a feature, such as the surface of a terrain feature (landscape, mountains, etc.), organ or other body part, digital object, etc.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of LADAVAC et al. into the combination of Kundu et al and Spivack et al. and Petrov et al. in order to reduced complexity such that it can be processed, transmitted, and/or rendered for display quickly and stored in reduced storage space compared to other representations of surfaces and objects (See LADAVAC et al. ¶ [0030]).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Kundu et al. (US 20230298226 A1) and Campbell et al. (US 20220255763 A1) and Petrov et al. (US 20230252736 A1) and further in view of Dougherty et al. (US 20100257252 A1).
As to claim 18, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the method according to claim 17 above. However, it does not expressly teach the details of wherein the subset of information includes a location and the rendering is generated based on a database of geospatial data for the location.
Dougherty et al., from analogous art, teaches wherein the subset of information includes a location and the rendering is generated based on a database of geospatial data for the location (See ¶¶ [0054]-[005], Teaches that in some embodiments, the database may comprise predetermined data such as feature descriptors and metadata associated with one or more landmarks. The predetermined data may be provided by the service provider. Additionally and optionally, the data may be user defined and transmitted by users. For example, landmarks that are not represented by pre-populated feature descriptors in the database may be represented by images provided by users. The term landmark may comprise any recognizable feature in an image, such as a textured portion of any object. For example, the blade of a windmill and the letter ‘G’ of an artist's signature in a wall painting might be two of the detected landmarks in the captured image of a room scene. When a pattern fails to be recognized by the image recognition engines, it may be determined that the pattern represents a new landmark and the user transmitted image may be used to represent the new landmark. In an embodiment, a user may decide that they desire to augment some space with content of their own choosing. For example, a user may enter an unknown area, collect information about the area such as feature descriptors, map data, and the like, and register the information in a database such that other users entering the area may then recognize the area and their place within the area. Additionally and optionally, the user or an application may choose to associate their own augmentation metadata with the area (e.g., placing virtual graffiti in the space) and make such data available to other users who may observe the area at the same or a different time. Multiple users may associate different metadata with a single area and allow the data to be accessible to different subsets of users. For example, a user may anchor some specific virtual content representing a small statue in a tavern, which may then be made visible to the user's on-line video game group when they enter the tavern while the virtual content may not be seen by any other mobile users in other video game groups. In another example, another user may have augmented the tavern with animated dancing animals. By enabling such augmentation and data sharing, the members of any type of gaming, social, or other type of group may share in the same set of common information about the tavern, its landmark descriptors, and their locations. At the same time, all users may not necessarily share in the same metadata associated with the venue.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Dougherty et al. into the combination of Kundu et al and Campbell et al. and Petrov et al. in order to create a merged user environment (See Dougherty et al. ¶ [0003]).
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Kundu et al. (US 20230298226 A1) and Campbell et al. (US 20220255763 A1) and Petrov et al. (US 20230252736 A1) and further in view of LADAVAC et al. (US 20230394767 A1).
As to claim 22, the combination of Kundu et al and Campbell et al. and Petrov et al. teaches the device according to claim 17 above. However, it does not expressly teach the details of wherein the rendering is a clay rendering.
LADAVAC et al., from analogous art, teaches wherein the rendering is a clay rendering (See ¶¶ [0030], [0059], Teaches that the mesh can be defined with sufficiently reduced complexity such that it can be processed, transmitted, and/or rendered for display quickly and stored in reduced storage space compared to other representations of surfaces and objects. For example, such a mesh is commonly used in applications such as efficient rendering of terrains, medical images, or objects modeled in a clay-like manner. Common applications include efficient rendering of terrains, medical images, or objects modeled in a clay-like manner. For example, the mesh can be used in the generation of a feature surface of a feature, such as the surface of a terrain feature (landscape, mountains, etc.), organ or other body part, digital object, etc.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of LADAVAC et al. into the combination of Kundu et al and Campbell et al. and Petrov et al. in order to reduced complexity such that it can be processed, transmitted, and/or rendered for display quickly and stored in reduced storage space compared to other representations of surfaces and objects (See LADAVAC et al. ¶ [0030]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Caswell et al. (US 20220292789 A1) teaches A cross reality system enables any of multiple devices to efficiently render shared location-based content. The cross reality system may include a cloud-based service that responds to requests from devices to localize with respect to a stored map. The service may return to the device information that localizes the device with respect to the stored map. In conjunction with localization information, the service may provide information about locations in the physical world proximate the device for which virtual content has been provided. Based on information received from the service, the device may render, or stop rendering, virtual content to each of multiple users based on the user's location and specified locations for the virtual content.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James R Hollister whose telephone number is (571)270-3152. The examiner can normally be reached Mon - Fri 7:30 am - 4:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Philip Chea can be reached at (571) 272-3951. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
James Hollister
/J.R.H./Examiner, Art Unit 2499 9/25/25
/PHILIP J CHEA/Supervisory Patent Examiner, Art Unit 2499
Prosecution Insights