DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation - 35 USC § 101
The limitations “based on the privacy settings, simultaneously enabling a presentation of an augmented viewing of the physical environment, such that during the simultaneous presentation, the first wearable extended reality appliance presents the first virtual object and the second virtual object in the physical environment, and the second wearable extended reality appliance presents the second virtual object, omitting presentation of the first virtual object in compliance with the privacy settings” are considered a practical application of presenting information based on a determination if it should be shown to a particular user of an augmented reality head set or not shown based on a privacy determination.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 10, 19, 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hwang et al. (US 2020/0004401)(Hereinafter referred to as Hwang).
Regarding claim 1, Hwang teaches A non-transitory computer readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform operations for managing privacy in an extended reality environment (Any of the steps, operations, or processes described herein may be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented with a computer program product comprising a computer-readable medium containing computer program code, which can be executed by a computer processor for performing any or all of the steps, operations, or processes described. See paragraph [0115])( Embodiments are directed to a near eye display (NED) system for displaying artificial reality content to a user. In some embodiments, multiple users may be in a local area, each using a different NED. A first user of a first NED may view virtual content using a first NED. The first NED may comprise an imaging device capable of capturing images of the local area, allowing the first NED to identify gestures performed by the first user and/or by other users in the local area. In some embodiments, the first NED may, in response to detecting one or more predetermined gestures performed by the first user, share virtual content displayed to the first user with a second user using a second NED, allowing the second user to view the virtual content through the second NED. See abstract), the operations comprising:
receiving image data from an image sensor associated with a wearable extended reality appliance, wherein the image data is reflective of a physical environment (The imaging device 315 may be used to capture a representation of the user's hands over time for use in tracking the user's hands ( e.g., by capturing multiple images per second of the user's hand). To achieve a more accurate capture, the imaging device 315 may be able to capture depth data of the local area or environment. This may be achieved by various means, such as by the use of computer vision algorithms that generate 3D data via detection of movement in the scene, by the emission of a grid pattern ( e.g., via emission of an infrared laser grid) and detection of depth from the variations in the reflection from the grid pattern, from computation of time-of-flight of reflected radiation (e.g., emitted infrared radiation that is reflected), and/or from the user of multiple cameras (e.g., binocular vision, stereophotogrammetry). The imaging device 315 may be positioned to capture a large spatial area, such that all hand movements within the spatial area are captured. In one embodiment, more than one imaging device 315 is used to capture images of the user's hands. As described in further detail below, the captured images of the user's hands may be used to identify various gestures for the user. Upon detecting these gestures in combination with the satisfaction of other conditions, a controller may execute certain associated actions. In another embodiment, the imaging device 315 may also capture images of one or more objects in the local area, and in particular the area encompassing the field of view of a user wearing an eyewear device that includes the NED 305. The imaging device 315 may also capture depth data of these one or more objects in the local area according to any of the methods described above. Although the imaging device 315 is illustrated in FIG. 3 as being separate from the NED 305, in some embodiments the imaging device is attached to the NED 305, e.g., attached to the frame 105, and may also be part of the DCA 340. The imaging device 315 may include one or more cameras, imaging sensors, one or more video cameras, any other device capable of capturing images, or some combination thereof. Additionally, the imaging device 315 may include one or more hardware and software filters ( e.g., used to increase signal to noise ratio). Image tracking data is communicated from the imaging device 315 to the controller 310, and the imaging device 315 receives one or more calibration parameters from the controller 310 to adjust one or more imaging parameters ( e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.). See paragraphs [0040-0043]);
accessing data characterizing a plurality of virtual objects for association with physical locations in the physical environment, the data representing a first virtual object and a second virtual object (In some embodiments, the transcription may be used to create or manipulate one or more content items, to display one or more virtual objects to the user of the NED 305, mapped to an executable command, and/or the like. See paragraph [0060])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064])(Any number of desired virtual objects by each of a plurality of local users);
accessing privacy settings classifying a first physical location of the first virtual object as private, classifying a first wearable extended reality appliance as approved for presentation of private information, and classifying a second wearable extended reality appliance as non-approved for presentation of the private information (In some embodiments, the transcription may be used to create or manipulate one or more content items, to display one or more virtual objects to the user of the NED 305, mapped to an executable command, and/or the like. See paragraph [0060]) (In some embodiments, instead of user profile information being stored in the user profile store 380, the NED system 300 may, using the communication module 375, access an online system such as a social network platform in order to access user profile information associated with different users. In some embodiments, the NED system 300 accesses the social network platform to identify social network connections between different users of the NED system 300. Based upon the social network connections, the NED system 300 may determine which types of content can be displayed to which users (e.g., content associated with a first user may be viewed by a second user having a "friend" relationship with the first user, but not by a third user having no social network connection to the first user). See paragraph [0063])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064])(Any number of desired virtual objects by each of a plurality of local users); and
based on the privacy settings, simultaneously enabling a presentation of an augmented viewing of the physical environment, such that during the simultaneous presentation, the first wearable extended reality appliance presents the first virtual object and the second virtual object in the physical environment, and the second wearable extended reality appliance presents the second virtual object, omitting presentation of the first virtual object in compliance with the privacy settings (In some embodiments, instead of user profile information being stored in the user profile store 380, the NED system 300 may, using the communication module 375, access an online system such as a social network platform in order to access user profile information associated with different users. In some embodiments, the NED system 300 accesses the social network platform to identify social network connections between different users of the NED system 300. Based upon the social network connections, the NED system 300 may determine which types of content can be displayed to which users (e.g., content associated with a first user may be viewed by a second user having a "friend" relationship with the first user, but not by a third user having no social network connection to the first user). See paragraph [0063])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064]).
Regarding claim 10, Hwang teaches the non-transitory computer readable medium of claim 1, wherein the operations further include presenting via the wearable extended reality appliance an indication that the first physical location associated with the first virtual object is classified as private (See figures 6A and 7, virtual object (unshared)).
Regarding claim 19, Hwang teaches A method for managing privacy in an extended reality environment ( Embodiments are directed to a near eye display (NED) system for displaying artificial reality content to a user. In some embodiments, multiple users may be in a local area, each using a different NED. A first user of a first NED may view virtual content using a first NED. The first NED may comprise an imaging device capable of capturing images of the local area, allowing the first NED to identify gestures performed by the first user and/or by other users in the local area. In some embodiments, the first NED may, in response to detecting one or more predetermined gestures performed by the first user, share virtual content displayed to the first user with a second user using a second NED, allowing the second user to view the virtual content through the second NED. See abstract), comprising:
receiving image data from an image sensor associated with a wearable extended reality appliance, wherein the image data is reflective of a physical environment (The imaging device 315 may be used to capture a representation of the user's hands over time for use in tracking the user's hands ( e.g., by capturing multiple images per second of the user's hand). To achieve a more accurate capture, the imaging device 315 may be able to capture depth data of the local area or environment. This may be achieved by various means, such as by the use of computer vision algorithms that generate 3D data via detection of movement in the scene, by the emission of a grid pattern ( e.g., via emission of an infrared laser grid) and detection of depth from the variations in the reflection from the grid pattern, from computation of time-of-flight of reflected radiation (e.g., emitted infrared radiation that is reflected), and/or from the user of multiple cameras (e.g., binocular vision, stereophotogrammetry). The imaging device 315 may be positioned to capture a large spatial area, such that all hand movements within the spatial area are captured. In one embodiment, more than one imaging device 315 is used to capture images of the user's hands. As described in further detail below, the captured images of the user's hands may be used to identify various gestures for the user. Upon detecting these gestures in combination with the satisfaction of other conditions, a controller may execute certain associated actions. In another embodiment, the imaging device 315 may also capture images of one or more objects in the local area, and in particular the area encompassing the field of view of a user wearing an eyewear device that includes the NED 305. The imaging device 315 may also capture depth data of these one or more objects in the local area according to any of the methods described above. Although the imaging device 315 is illustrated in FIG. 3 as being separate from the NED 305, in some embodiments the imaging device is attached to the NED 305, e.g., attached to the frame 105, and may also be part of the DCA 340. The imaging device 315 may include one or more cameras, imaging sensors, one or more video cameras, any other device capable of capturing images, or some combination thereof. Additionally, the imaging device 315 may include one or more hardware and software filters ( e.g., used to increase signal to noise ratio). Image tracking data is communicated from the imaging device 315 to the controller 310, and the imaging device 315 receives one or more calibration parameters from the controller 310 to adjust one or more imaging parameters ( e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.). See paragraphs [0040-0043]);
accessing data characterizing a plurality of virtual objects for association with physical locations in the physical environment, the data representing a first virtual object and a second virtual object (In some embodiments, the transcription may be used to create or manipulate one or more content items, to display one or more virtual objects to the user of the NED 305, mapped to an executable command, and/or the like. See paragraph [0060])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064])(Any number of desired virtual objects by each of a plurality of local users);
accessing privacy settings classifying a first physical location of the first virtual object as private, classifying a first wearable extended reality appliance as approved for presentation of private information, and classifying a second wearable extended reality appliance as non-approved for presentation of the private information (In some embodiments, the transcription may be used to create or manipulate one or more content items, to display one or more virtual objects to the user of the NED 305, mapped to an executable command, and/or the like. See paragraph [0060]) (In some embodiments, instead of user profile information being stored in the user profile store 380, the NED system 300 may, using the communication module 375, access an online system such as a social network platform in order to access user profile information associated with different users. In some embodiments, the NED system 300 accesses the social network platform to identify social network connections between different users of the NED system 300. Based upon the social network connections, the NED system 300 may determine which types of content can be displayed to which users (e.g., content associated with a first user may be viewed by a second user having a "friend" relationship with the first user, but not by a third user having no social network connection to the first user). See paragraph [0063])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064])(Any number of desired virtual objects by each of a plurality of local users); and
based on the privacy settings, simultaneously enabling a presentation of an augmented viewing of the physical environment, such that during the simultaneous presentation, the first wearable extended reality appliance presents the first virtual object and the second virtual object in the physical environment, and the second wearable extended reality appliance presents the second virtual object, omitting presentation of the first virtual object in compliance with the privacy settings (In some embodiments, instead of user profile information being stored in the user profile store 380, the NED system 300 may, using the communication module 375, access an online system such as a social network platform in order to access user profile information associated with different users. In some embodiments, the NED system 300 accesses the social network platform to identify social network connections between different users of the NED system 300. Based upon the social network connections, the NED system 300 may determine which types of content can be displayed to which users (e.g., content associated with a first user may be viewed by a second user having a "friend" relationship with the first user, but not by a third user having no social network connection to the first user). See paragraph [0063])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064]).
Regarding claim 20, Hwang teaches A system for managing privacy in an extended reality environment ( Embodiments are directed to a near eye display (NED) system for displaying artificial reality content to a user. In some embodiments, multiple users may be in a local area, each using a different NED. A first user of a first NED may view virtual content using a first NED. The first NED may comprise an imaging device capable of capturing images of the local area, allowing the first NED to identify gestures performed by the first user and/or by other users in the local area. In some embodiments, the first NED may, in response to detecting one or more predetermined gestures performed by the first user, share virtual content displayed to the first user with a second user using a second NED, allowing the second user to view the virtual content through the second NED. See abstract), comprising:
at least one processing device configured (FIG. 3 is a block diagram of a NED system 300, in accordance with one or more embodiments-. The NED system 300 shown by FIG. 3 comprises a NED 305 coupled to a controller 310, with the controller 310 coupled to an imaging device 315. While FIG. 3 shows an example NED system 300 including one NED 305 and one imaging device 315, in other embodiments any number of these components may be included in the NED system 300. See paragraph [0028]) to:
receive image data from an image sensor associated with a wearable extended reality appliance, wherein the image data is reflective of a physical environment (The imaging device 315 may be used to capture a representation of the user's hands over time for use in tracking the user's hands ( e.g., by capturing multiple images per second of the user's hand). To achieve a more accurate capture, the imaging device 315 may be able to capture depth data of the local area or environment. This may be achieved by various means, such as by the use of computer vision algorithms that generate 3D data via detection of movement in the scene, by the emission of a grid pattern ( e.g., via emission of an infrared laser grid) and detection of depth from the variations in the reflection from the grid pattern, from computation of time-of-flight of reflected radiation (e.g., emitted infrared radiation that is reflected), and/or from the user of multiple cameras (e.g., binocular vision, stereophotogrammetry). The imaging device 315 may be positioned to capture a large spatial area, such that all hand movements within the spatial area are captured. In one embodiment, more than one imaging device 315 is used to capture images of the user's hands. As described in further detail below, the captured images of the user's hands may be used to identify various gestures for the user. Upon detecting these gestures in combination with the satisfaction of other conditions, a controller may execute certain associated actions. In another embodiment, the imaging device 315 may also capture images of one or more objects in the local area, and in particular the area encompassing the field of view of a user wearing an eyewear device that includes the NED 305. The imaging device 315 may also capture depth data of these one or more objects in the local area according to any of the methods described above. Although the imaging device 315 is illustrated in FIG. 3 as being separate from the NED 305, in some embodiments the imaging device is attached to the NED 305, e.g., attached to the frame 105, and may also be part of the DCA 340. The imaging device 315 may include one or more cameras, imaging sensors, one or more video cameras, any other device capable of capturing images, or some combination thereof. Additionally, the imaging device 315 may include one or more hardware and software filters ( e.g., used to increase signal to noise ratio). Image tracking data is communicated from the imaging device 315 to the controller 310, and the imaging device 315 receives one or more calibration parameters from the controller 310 to adjust one or more imaging parameters ( e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.). See paragraphs [0040-0043]);
access data characterizing a plurality of virtual objects for association with physical locations in the physical environment, the data representing a first virtual object and a second virtual object (In some embodiments, the transcription may be used to create or manipulate one or more content items, to display one or more virtual objects to the user of the NED 305, mapped to an executable command, and/or the like. See paragraph [0060])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064])(Any number of desired virtual objects by each of a plurality of local users);
access privacy settings classifying a first physical location of the first virtual object as private, classifying a first wearable extended reality appliance as approved for presentation of private information, and classifying a second wearable extended reality appliance as non-approved for presentation of the private information (In some embodiments, the transcription may be used to create or manipulate one or more content items, to display one or more virtual objects to the user of the NED 305, mapped to an executable command, and/or the like. See paragraph [0060]) (In some embodiments, instead of user profile information being stored in the user profile store 380, the NED system 300 may, using the communication module 375, access an online system such as a social network platform in order to access user profile information associated with different users. In some embodiments, the NED system 300 accesses the social network platform to identify social network connections between different users of the NED system 300. Based upon the social network connections, the NED system 300 may determine which types of content can be displayed to which users (e.g., content associated with a first user may be viewed by a second user having a "friend" relationship with the first user, but not by a third user having no social network connection to the first user). See paragraph [0063])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064])(Any number of desired virtual objects by each of a plurality of local users); and
based on the privacy settings, simultaneously enable a presentation of an augmented viewing of the physical environment, such that during the simultaneous presentation, the first wearable extended reality appliance presents the first virtual object and the second virtual object in the physical environment, and the second wearable extended reality appliance presents the second virtual object, omitting presentation of the first virtual object in compliance with the privacy settings (In some embodiments, instead of user profile information being stored in the user profile store 380, the NED system 300 may, using the communication module 375, access an online system such as a social network platform in order to access user profile information associated with different users. In some embodiments, the NED system 300 accesses the social network platform to identify social network connections between different users of the NED system 300. Based upon the social network connections, the NED system 300 may determine which types of content can be displayed to which users (e.g., content associated with a first user may be viewed by a second user having a "friend" relationship with the first user, but not by a third user having no social network connection to the first user). See paragraph [0063])( In some embodiments, different users may concurrently use the NED system 300 via different NEDs 305. For example, the controller 310 may transmit display instructions to a first NED 305 to display a first AR environment to a first user, and to a second NED to display a second AR environment to a second user. As such, each of the first and second users may be able to view different virtual objects through their respective AR environments. In some embodiments, the NED system 300 may cause different content to be displayed in the first or second AR environments in response to detecting one or more predetermined gestures performed by the first and/or second users. For example, as discussed in greater detail below, the NED system 300 may, in response to detecting a particular gesture by the first user, display to the second user a visual flair emphasizing the gesture in the second AR environment. In some embodiments, the NED system 300 may, in response to a gesture by the first user, change the permissions of a content item such that the content item can be viewed by the second user in the second AR environment. See paragraph [0064]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 2-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang et al. (US 2020/0004401)(Hereinafter referred to as Hwang) in view of Spivack et al. (US 2019/0108578)(Hereinafter referred to as Spivack).
Regarding claim 2, Hwang teaches The non-transitory computer readable medium of claim 1, but is silent to wherein the operations further include designating other virtual objects in the first physical location as private and blocking the other virtual objects from display via the second wearable extended reality appliance.
Spivack teaches creating virtual note objects which can be made private for individual users (For example, a virtual object that is a private note can be posted as a world-locked object—for example the VOB can be sent to a user Bob such that it appears in the doorway of their house in a locked position as a sticky note or other type of AR message to Bob. In addition, Bob can get alerted when he is near a place with a private note to him. Private notes can be locked to positions in the world for people so they see them when they are there. They can also be timed to start and end at certain dates and times, or only when certain conditions are met (not on a weekend, when you are about to go grocery shopping, etc.). Other users may be able to see it when in that place, dependent on permissions and other contextual parameters. See paragraph [0368]).
Hwang and Spivack teach of the ability to present only certain information to users and Spivack teaches that a user can create individual note virtual objects to be viewed by another user at a specific location, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the virtual note capabilities of Spivack such that the user could send individualized virtual notes to friends.
Regarding claim 3, Hwang teaches The non-transitory computer readable medium of claim 1, but is silent to wherein the operations further include detecting a user-initiated movement of the first virtual object to a second physical location, and in response to the user-initiated movement, enabling the second wearable extended reality appliance to present the first virtual object and the second virtual object.
Spivack teaches creating virtual note objects which can be made private for individual users (For example, a virtual object that is a private note can be posted as a world-locked object—for example the VOB can be sent to a user Bob such that it appears in the doorway of their house in a locked position as a sticky note or other type of AR message to Bob. In addition, Bob can get alerted when he is near a place with a private note to him. Private notes can be locked to positions in the world for people so they see them when they are there. They can also be timed to start and end at certain dates and times, or only when certain conditions are met (not on a weekend, when you are about to go grocery shopping, etc.). Other users may be able to see it when in that place, dependent on permissions and other contextual parameters. See paragraph [0368])( The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184]).
Hwang and Spivack teach of the ability to present only certain information to users and Spivack teaches that a user can create individual note virtual objects to be viewed by another user at a specific location, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the virtual note capabilities of Spivack such that the user could send individualized virtual notes to friends.
Regarding claim 4, Hwang teaches The non-transitory computer readable medium of claim 1, but is silent to wherein the operations further include detecting a user-initiated movement of the second virtual object to the first physical location, and in response to the user-initiated movement, preventing the second wearable extended reality appliance from presenting the second virtual object.
Spivack teaches creating virtual note objects which can be made private for individual users (For example, a virtual object that is a private note can be posted as a world-locked object—for example the VOB can be sent to a user Bob such that it appears in the doorway of their house in a locked position as a sticky note or other type of AR message to Bob. In addition, Bob can get alerted when he is near a place with a private note to him. Private notes can be locked to positions in the world for people so they see them when they are there. They can also be timed to start and end at certain dates and times, or only when certain conditions are met (not on a weekend, when you are about to go grocery shopping, etc.). Other users may be able to see it when in that place, dependent on permissions and other contextual parameters. See paragraph [0368])( The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184]).
Hwang and Spivack teach of the ability to present only certain information to users and Spivack teaches that a user can create individual note virtual objects to be viewed by another user at a specific location, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the virtual note capabilities of Spivack such that the user could send individualized virtual notes to friends.
Regarding claim 5, Hwang teaches The non-transitory computer readable medium of claim 1, but is silent to wherein the privacy settings further classify the first virtual object as private, and the operations further including detecting a user-initiated movement of the first virtual object to a second physical location, and in response to the user-initiated movement, preventing the second wearable extended reality appliance from presenting the first virtual object at the second physical location.
Spivack teaches creating virtual note objects which can be made private for individual users (For example, a virtual object that is a private note can be posted as a world-locked object—for example the VOB can be sent to a user Bob such that it appears in the doorway of their house in a locked position as a sticky note or other type of AR message to Bob. In addition, Bob can get alerted when he is near a place with a private note to him. Private notes can be locked to positions in the world for people so they see them when they are there. They can also be timed to start and end at certain dates and times, or only when certain conditions are met (not on a weekend, when you are about to go grocery shopping, etc.). Other users may be able to see it when in that place, dependent on permissions and other contextual parameters. See paragraph [0368])( The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184]).
Hwang and Spivack teach of the ability to present only certain information to users and Spivack teaches that a user can create individual note virtual objects to be viewed by another user at a specific location, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the virtual note capabilities of Spivack such that the user could send individualized virtual notes to friends.
Regarding claim 6, Hwang teaches The non-transitory computer readable medium of claim 1, but is silent to wherein the operations further include detecting a user-initiated movement of the second virtual object to the first physical location, and enabling the second wearable extended reality appliance to present the second virtual object at the first location.
Spivack teaches creating virtual note objects which can be made private for individual users (For example, a virtual object that is a private note can be posted as a world-locked object—for example the VOB can be sent to a user Bob such that it appears in the doorway of their house in a locked position as a sticky note or other type of AR message to Bob. In addition, Bob can get alerted when he is near a place with a private note to him. Private notes can be locked to positions in the world for people so they see them when they are there. They can also be timed to start and end at certain dates and times, or only when certain conditions are met (not on a weekend, when you are about to go grocery shopping, etc.). Other users may be able to see it when in that place, dependent on permissions and other contextual parameters. See paragraph [0368])( The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184]).
Hwang and Spivack teach of the ability to present only certain information to users and Spivack teaches that a user can create individual note virtual objects to be viewed by another user at a specific location, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the virtual note capabilities of Spivack such that the user could send individualized virtual notes to friends.
Regarding claim 7, Hwang teachse the non-transitory computer readable medium of claim 1, but is silent to wherein the privacy settings further classify a second physical location of the second virtual object as public.
Spivack teaches creating virtual note objects which can be made private for individual users or public based on need (For example, a virtual object that is a private note can be posted as a world-locked object—for example the VOB can be sent to a user Bob such that it appears in the doorway of their house in a locked position as a sticky note or other type of AR message to Bob. In addition, Bob can get alerted when he is near a place with a private note to him. Private notes can be locked to positions in the world for people so they see them when they are there. They can also be timed to start and end at certain dates and times, or only when certain conditions are met (not on a weekend, when you are about to go grocery shopping, etc.). Other users may be able to see it when in that place, dependent on permissions and other contextual parameters. See paragraph [0368])( The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184]).
Hwang and Spivack teach of the ability to present only certain information to users and Spivack teaches that a user can create individual note virtual objects to be viewed by another user at a specific location, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the virtual note capabilities of Spivack such that the user could send individualized virtual notes to friends or public information to a community.
Regarding claim 8, Hwang in view of Spivack teaches The non-transitory computer readable medium of claim 7, wherein the operations further include detecting a user-initiated movement of the first virtual object to the second physical location, and in response to the user-initiated movement, enabling the second wearable extended reality appliance to present the first virtual object (Spivack; For example, a virtual object that is a private note can be posted as a world-locked object—for example the VOB can be sent to a user Bob such that it appears in the doorway of their house in a locked position as a sticky note or other type of AR message to Bob. In addition, Bob can get alerted when he is near a place with a private note to him. Private notes can be locked to positions in the world for people so they see them when they are there. They can also be timed to start and end at certain dates and times, or only when certain conditions are met (not on a weekend, when you are about to go grocery shopping, etc.). Other users may be able to see it when in that place, dependent on permissions and other contextual parameters. See paragraph [0368])( Spivack; The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184]).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang et al. (US 2020/0004401)(Hereinafter referred to as Hwang) in view of Lemay et al. (US 2021/0191600)(Hereinafter referred to as Lemay).
Regarding claim 9, Hwang teaches the non-transitory computer readable medium of claim 1, but is silent to wherein enabling the simultaneous presentation includes causing the second wearable extended reality appliance to present the second virtual object at a second physical location associated with the second virtual object and to present a distorted representation of the first virtual object at the first physical location of the first virtual object in compliance with the privacy settings.
Lemay teaches presenting distorted representations of private virtual objects (In some embodiments, forgoing displaying at least a portion of the user interface object that is associated with the second user includes displaying the second view of the three-dimensional environment with an indication of the user interface object (e.g., a shape or outline of the user interface object) (e.g., indication 7216 for the first view 7236, or indication 7204 for the first view 7036 in FIG. 7F) in the second view of the three-dimensional environment, where at least a portion of content in the user interface object (e.g., first portion of the user interface object) is obscured (e.g., dimmed or blurred by a visual effect) in the indication of the user interface object (e.g., the second view of the three-dimensional environment as displayed via the first display generation component shows an outline or a translucent image of the user interface object at the position of the user interface object in the respective view of the three-dimensional environment displayed via the second display generation component, but the outline or translucent image of the user interface object displayed via the first display generation component does not include the private content that is shown on the user interface object in the respective view of the three-dimensional environment displayed via the second display generation component). See paragraph [0306]).
Hwang and Lemay teach of presenting object to users and Lemay teaches that the object may be distorted if it is private, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the privacy distortion techniques of Lemay such that the virtual objects could be obscured to prevent the other user from viewing information not intended for them to see.
Claim(s) 11-15, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hwang et al. (US 2020/0004401)(Hereinafter referred to as Hwang) in view of Zarepour et al. (“A Context-based Privacy Preserving Framework for Wearable Visual Lifeloggers”, IEEE, 2016.)(Hereinafter referred to as Zarepour).
Regarding claim 11, Hwang teaches the non-transitory computer readable medium of claim 1, but is silent to wherein the operations further include analyzing the image data using an object detection algorithm to detect a physical object in a particular region of the physical environment, and classifying the particular region of the physical environment as public based on the detected physical object.
Zarepour teaches the ability to detect objects in the view based on image analysis and blur them depending on if they are deemed private/sensitive (The ability of wearable cameras to continuously capture the first person viewpoint with minimal user interaction, has made them very attractive in many application domains. Wearable technology today is available and useful but not widely used and accepted due to various challenges mainly privacy concerns. In this paper, we introduce a novel efficient privacy aware framework for wearable cameras that can protect all sensitive subjects such as people, objects (e.g, display screens, license plates and credit cards) and locations (e.g, bathrooms and bedrooms). It uses the contextual information obtained from the wearable’s sensors and recorded images to identify the potential sensitive subjects in each image. Using image processing techniques, first the sensitive subjects are recognized and then blurred or eliminated. Finally the edited image is ready to publish. Using a practical case study, we experimentally evaluate the proposed framework. Our experimental results show that the proposed system can detect and blur the sensitive subjects among around 300 images with around 70% accuracy. See abstract).
Hwang and Zarepour teach of head mounted wearables and Zarepour teaches that by identifying real objects in videos the system can use a privacy aware framework to obscure sensitive data detected, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the privacy aware framework to protect sensitive objects in the view such that sensitive information in each of the frames can be obscured from third party sources.
Regarding claim 12, Hwang teaches the non-transitory computer readable medium of claim 1, but is silent to wherein the operations further include receiving an input identifying a specific region of the physical environment as private.
Zarepour teaches the ability to detect objects in the view based on image analysis and blur them depending on if they are deemed private/sensitive (input is image data captured by external sensor) (The ability of wearable cameras to continuously capture the first person viewpoint with minimal user interaction, has made them very attractive in many application domains. Wearable technology today is available and useful but not widely used and accepted due to various challenges mainly privacy concerns. In this paper, we introduce a novel efficient privacy aware framework for wearable cameras that can protect all sensitive subjects such as people, objects (e.g, display screens, license plates and credit cards) and locations (e.g, bathrooms and bedrooms). It uses the contextual information obtained from the wearable’s sensors and recorded images to identify the potential sensitive subjects in each image. Using image processing techniques, first the sensitive subjects are recognized and then blurred or eliminated. Finally the edited image is ready to publish. Using a practical case study, we experimentally evaluate the proposed framework. Our experimental results show that the proposed system can detect and blur the sensitive subjects among around 300 images with around 70% accuracy. See abstract).
Hwang and Zarepour teach of head mounted wearables and Zarepour teaches that by identifying real objects in videos the system can use a privacy aware framework to obscure sensitive data detected, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Hwang with the privacy aware framework to protect sensitive objects in the view such that sensitive information in each of the frames can be obscured from third party sources.
Regarding claim 13, Hwang in view of Zarepour teaches the non-transitory computer readable medium of claim 12, wherein the received input includes the image data and the operations further include analyzing the image data using an object detection algorithm to detect a physical object in the specific region of the physical environment, and identifying the specific region of the physical environment as private based on the detected physical object (Zarepour; input is image data captured by external sensor) (Zarepour; The ability of wearable cameras to continuously capture the first person viewpoint with minimal user interaction, has made them very attractive in many application domains. Wearable technology today is available and useful but not widely used and accepted due to various challenges mainly privacy concerns. In this paper, we introduce a novel efficient privacy aware framework for wearable cameras that can protect all sensitive subjects such as people, objects (e.g, display screens, license plates and credit cards) and locations (e.g, bathrooms and bedrooms). It uses the contextual information obtained from the wearable’s sensors and recorded images to identify the potential sensitive subjects in each image. Using image processing techniques, first the sensitive subjects are recognized and then blurred or eliminated. Finally the edited image is ready to publish. Using a practical case study, we experimentally evaluate the proposed framework. Our experimental results show that the proposed system can detect and blur the sensitive subjects among around 300 images with around 70% accuracy. See abstract).
Regarding claim 14, Hwang in view of Zarepour teaches the non-transitory computer readable medium of claim 13, wherein the physical object is movable, and the operations further include analyzing the image data to identify a movement of the physical object out of the specific region of the physical environment, and reclassifying the specific region of the physical environment as public in response to the movement of the physical object (movable could be person or vehicle with license plate and it will sensitive information will no longer be obscured because it is not present) (Zarepour; The ability of wearable cameras to continuously capture the first person viewpoint with minimal user interaction, has made them very attractive in many application domains. Wearable technology today is available and useful but not widely used and accepted due to various challenges mainly privacy concerns. In this paper, we introduce a novel efficient privacy aware framework for wearable cameras that can protect all sensitive subjects such as people, objects (e.g, display screens, license plates and credit cards) and locations (e.g, bathrooms and bedrooms). It uses the contextual information obtained from the wearable’s sensors and recorded images to identify the potential sensitive subjects in each image. Using image processing techniques, first the sensitive subjects are recognized and then blurred or eliminated. Finally the edited image is ready to publish. Using a practical case study, we experimentally evaluate the proposed framework. Our experimental results show that the proposed system can detect and blur the sensitive subjects among around 300 images with around 70% accuracy. See abstract).
Regarding claim 15, Hwang in view of Zarepour teaches The non-transitory computer readable medium of claim 12, wherein the operations further include classifying the first virtual object as private when a distance of the first virtual object from the specific region of the physical environment is less than a threshold distance (Zarepour; The ability of wearable cameras to continuously capture the first person viewpoint with minimal user interaction, has made them very attractive in many application domains. Wearable technology today is available and useful but not widely used and accepted due to various challenges mainly privacy concerns. In this paper, we introduce a novel efficient privacy aware framework for wearable cameras that can protect all sensitive subjects such as people, objects (e.g, display screens, license plates and credit cards) and locations (e.g, bathrooms and bedrooms). It uses the contextual information obtained from the wearable’s sensors and recorded images to identify the potential sensitive subjects in each image. Using image processing techniques, first the sensitive subjects are recognized and then blurred or eliminated. Finally the edited image is ready to publish. Using a practical case study, we experimentally evaluate the proposed framework. Our experimental results show that the proposed system can detect and blur the sensitive subjects among around 300 images with around 70% accuracy. See abstract) (Zarepour; In cases of images from toilets or bathroom, the results from the environment detection were used and the whole image is blurred. Three final images after blurring phases are presented in Fig. 6.).
Regarding claim 18, Hwang in view of Zarepour teaches The non-transitory computer readable medium of claim 12, wherein the received input includes the image data and the operations further include analyzing the image data to determine a relation between two physical objects in the specific region of the physical environment, and identifying the specific region of the physical environment as private based on the determined relation between the two physical objects (Zarepour; The ability of wearable cameras to continuously capture the first person viewpoint with minimal user interaction, has made them very attractive in many application domains. Wearable technology today is available and useful but not widely used and accepted due to various challenges mainly privacy concerns. In this paper, we introduce a novel efficient privacy aware framework for wearable cameras that can protect all sensitive subjects such as people, objects (e.g, display screens, license plates and credit cards) and locations (e.g, bathrooms and bedrooms). It uses the contextual information obtained from the wearable’s sensors and recorded images to identify the potential sensitive subjects in each image. Using image processing techniques, first the sensitive subjects are recognized and then blurred or eliminated. Finally the edited image is ready to publish. Using a practical case study, we experimentally evaluate the proposed framework. Our experimental results show that the proposed system can detect and blur the sensitive subjects among around 300 images with around 70% accuracy. See abstract) (Zarepour; In cases of images from toilets or bathroom, the results from the environment detection were used and the whole image is blurred. Three final images after blurring phases are presented in Fig. 6.).
Allowable Subject Matter
Claims 16 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Hwang in view of Zarepour is the closest prior art of record.
Hwang teaches obscuring virtual objects taken by wearables (Embodiments are directed to a near eye display (NED) system for displaying artificial reality content to a user. In some embodiments, multiple users may be in a local area, each using a different NED. A first user of a first NED may view virtual content using a first NED. The first NED may comprise an imaging device capable of capturing images of the local area, allowing the first NED to identify gestures performed by the first user and/or by other users in the local area. In some embodiments, the first NED may, in response to detecting one or more predetermined gestures performed by the first user, share virtual content displayed to the first user with a second user using a second NED, allowing the second user to view the virtual content through the second NED. See abstract) and
Zarepour teaches obscuring physical objects in images taken by wearables (The ability of wearable cameras to continuously capture the first person viewpoint with minimal user interaction, has made them very attractive in many application domains. Wearable technology today is available and useful but not widely used and accepted due to various challenges mainly privacy concerns. In this paper, we introduce a novel efficient privacy aware framework for wearable cameras that can protect all sensitive subjects such as people, objects (e.g, display screens, license plates and credit cards) and locations (e.g, bathrooms and bedrooms). It uses the contextual information obtained from the wearable’s sensors and recorded images to identify the potential sensitive subjects in each image. Using image processing techniques, first the sensitive subjects are recognized and then blurred or eliminated. Finally the edited image is ready to publish. Using a practical case study, we experimentally evaluate the proposed framework. Our experimental results show that the proposed system can detect and blur the sensitive subjects among around 300 images with around 70% accuracy. See abstract)
Regarding claim 16, Hwang in view of Zarepour teaches the non-transitory computer readable medium of claim 12, but is silent to wherein the operations further include classifying the first virtual object as private when the first virtual object is docked to a physical object included in the specific region of the physical environment.
The prior art of record alone or in combination is silent to the limitations “wherein the operations further include classifying the first virtual object as private when the first virtual object is docked to a physical object included in the specific region of the physical environment.” Of claim 16 when read in light of the rest of the limitations in claim 16 and thus claim 16 contains allowable subject matter.
Claims 17 contains allowable subject matter because it depends on a claim that contains allowable subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Hubbard (US 2022/0303301), generally relates to controlling access based on privacy for users (See paragraph [0144]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS R WILSON whose telephone number is (571)272-0936. The examiner can normally be reached M-F 7:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (572)-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS R WILSON/Primary Examiner, Art Unit 2611
Prosecution Insights