DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/30/2026 has been entered.
Response to Arguments
Applicant’s arguments, see page 8-9, filed 3/30/2026 with respect to 35 USC §103 rejection of claims 1-20 have been fully considered but are moot because the arguments do not apply to any of the references being used in the current rejection and the arguments were directed towards the claims as amended. Applicant has amended claims and argues previously cited references fails to disclose amended claim limitations. In response, examiner points to newly cited Mei to teach amended claim limitations.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lei et al (US 20200404159 A1) in view of Swaminathan et al (US 20140168056 A1) and Mei (US 20190035293 A1).
Regarding claim 1, Lei discloses a wearable system ([0027] mixed-reality environment supported on an HMD device), comprising:
a head-mounted display ([0027] user can employ the HMD device);
a set of sensors ([0034] HMD device can include many types of sensors); and
a processor ([0030] HMD device may have one or more processors) configured to:
determine whether an activation criteria has been met ([0049] auto-focus subsystem may be configured with one or more criteria that, upon satisfaction or dissatisfaction, are determinative as to whether the HMD device triggers or suppresses an auto-focus operation.);
in response to the activation criteria being met, enter a focus mode that reduces distraction in an extended reality environment ([0050] auto-focus operation may comprise the auto-focus subsystem automatically focusing on the content within the ROI of the display FOV); and
while in the focus mode:
receive imaging data of a physical environment using the set of sensors ([0041] composite rendering is substantially similar to what the local user views through the see-through mixed-reality display on the local HMD device);
identify, using the imaging data, an object that is positioned within a first region of the physical environment and that meets a distraction criteria ([0050] the user is using his hands in a manner in which the user wishes to clearly view his hands and the hands are a focus point for the user within the ROI);
display the extended reality environment using the head-mounted display ([0039] physical environment in which the user interacts with holographic virtual objects that are viewable by the user through the see-through mixed-reality display system on the HMD device).
Swaminathan discloses select additional content based on the identified object ([0039] augmented reality information may be determined, for example, by application of active segmentation and object recognition techniques to obtain electronic representations of the target areas and/or object);
generate the extended reality environment that includes a reproduction of the first region of the physical environment ([0040] presentation region for a media content item may be deployed to an augmented reality environment by a user for the purpose of defining a location (e.g., a position and/or orientation) where the media content item is presented within that augmented reality environment),
wherein the reproduction includes a subregion in which the imaging data associated with the identified object is ([0003] Such supplemental information may form an information layer that overlays real-world objects that are captured in a camera view of the mobile device, [0060] determine an image segment, perform image recognition on the segment, and present any AR information that is associated with the image recognition, [0061] the AR information can be an AR object tag which represents additional data displayed over the image [0078] the object tag 504-1 can be hidden (i.e., a visible property can be set to false). The transition between the show and hide state of the object tag can be a fade, dissolve, swipe, or other graphical transition)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include select additional content based on the identified object; generate the extended reality environment that includes a reproduction of the first region of the physical environment, wherein the reproduction includes a subregion in which the imaging data associated with the identified object is removed from the extended reality environment and replaced with the additional content as described by Swaminathan.
The motivation for doing so would have been for displaying an image to a user of a scene viewable by the user, receiving information indicative of an eye gaze of the user, determining an area of interest within the image based on the eye gaze information, causing an object tag to be displayed in an image of the scene based on the location of the area of interest, and displaying augmented reality information if the area of interest lingers on or about the object tag (Swaminathan, [0005]).
Mei discloses identify, using the imaging data, an object that is positioned within a first region of the physical environment and that meets a distraction criteria ([0032] the system determines whether the patient's gaze has a correlation with the primary object 210 above a threshold attention criterion);
wherein the reproduction includes a subregion in which the imaging data associated with the identified object is removed from the extended reality environment ([0031] when a threshold distraction criterion is exceeded, the system removes the correlated secondary object from the displayed scene)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include identify, using the imaging data, an object that is positioned within a first region of the physical environment and that meets a distraction criteria, wherein the reproduction includes a subregion in which the imaging data associated with the identified object is removed from the extended reality environment as described by Mei.
The motivation for doing so would have been to reduce the complexity of the displayed scene. (Mei, [0004]).
Lei, Swaminathan and Mei are combinable because they are from the same field of invention.
Therefore, it would have been obvious to combine Lei, Swaminathan and Mei to obtain the invention as specified in claim 1.
Regarding claim 2, Lei discloses determine a current activity of a user ([0045] ROI may be located at a central region of the display system FOV as that is a likely location at which the user gazes);
determine a focus level associated with the current activity ([0045] During interactions with holograms of the virtual objects, the hands of the user may be located within the ROI which is defined by a set of spherical coordinates.); and
select the distraction criteria based on the determined focus level ([0046] auto-focus subsystem collectively utilizes this data to control the auto-focus operations to reduce the auto-focus hunting effect created by hand movements during interactions with the rendered holographic virtual objects.).
Regarding claim 3, Lei discloses determine whether the activation criteria has been met based on the current activity ([0049] auto-focus subsystem may be configured with one or more criteria that, upon satisfaction or dissatisfaction, are determinative as to whether the HMD device triggers or suppresses an auto-focus operation.).
Regarding claim 4, Lei discloses the additional content comprises a virtual environment that represents the physical environment with the identified object removed ([0027] virtual objects in the FOV coexist with real objects in the three-dimensional (3D) physical environment to create the mixed-reality experience).
Regarding claim 5, Lei discloses the additional content comprises a virtual object ([0045] during interactions with holograms of the virtual objects,)
Regarding claim 6, Lei discloses the virtual object is generated using the imaging data of the physical environment associated with the object ([0060] , the hologram comprising a virtual object that is located in the physical environment at a known location.).
Regarding claim 7, Lei discloses determining whether the activation criteria has been met comprises: detecting an attentive state of the user ([0052] auto-focus subsystem continues to operate as normal by auto-focusing on the environment detected in the ROI); and
determining whether the attentive state meets an attention threshold ([0054] Auto-focus operations may be disabled when one or more hands are not statically located for a duration of time that satisfies a pre-set threshold limit (e.g., 3 seconds)).
Regarding claim 8, Lei discloses wherein the processor is configured to: determine an attentive state of the user ([0052] auto-focus subsystem continues to operate as normal by auto-focusing on the environment detected in the ROI); and
select the distraction criteria based on the determined attentive state ([0054] auto-focus operations can be executed when one or more hands are statically located in the ROI).
Regarding claim 9, Lei is silent to wherein selecting the additional content based on the identified object comprises:
determining an object type of the identified object; and
selecting the additional content using a stored user preference for the determined object type.
Swaminathan discloses wherein selecting the additional content based on the identified object comprises:
determining an object type of the identified object ([0039] object recognition pipeline can be based on an area of interest defined by the user's eye gaze); and
selecting the additional content using a stored user preference for the determined object type ([0039] Augmented reality information may be determined, for example, by application of active segmentation and object recognition techniques to obtain electronic representations of the target areas and/or objects)
Lei, Swaminathan and Mei are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include wherein selecting the additional content based on the identified object comprises: determining an object type of the identified object; and selecting the additional content using a stored user preference for the determined object type as described by Swaminathan.
The motivation for doing so would have been for displaying an image to a user of a scene viewable by the user, receiving information indicative of an eye gaze of the user, determining an area of interest within the image based on the eye gaze information, causing an object tag to be displayed in an image of the scene based on the location of the area of interest, and displaying augmented reality information if the area of interest lingers on or about the object tag (Swaminathan, [0005]).
Therefore, it would have been obvious to combine Lei, Swaminathan and Mei to obtain the invention as specified in claim 9.
Regarding claim 10, Lei discloses a head-mounted device to be worn by a user ([0027] mixed-reality environment supported on an HMD device),, comprising:
a display ([0027] user can employ the HMD device);
a first set of sensors configured to detect a physical environment around the user ([0034] HMD device can include many types of sensors);
a second set of sensors configured to measure one or more physiological parameters of the user ([0080] a gaze detection subsystem configured for detecting a direction of gaze of each eye of a user or a direction or location of focus); and
a processor ([0030] HMD device may have one or more processors) configured to:
generate an extended reality environment that includes a first reproduction of a first portion of the physical environment;
display the extended reality environment using the display ([0039] holographic virtual objects that are viewable by the user through the see-through mixed-reality display system on the HMD device);
detect, using the second set of sensors, user engagement with a candidate object in the first reproduction ([0049] auto-focus subsystem may be configured with one or more criteria that, upon satisfaction or dissatisfaction, are determinative as to whether the HMD device triggers or suppresses an auto-focus operation.);
in response to the detected user engagement meeting a selection criteria, assigning a distraction type to the candidate object ([0050] auto-focus operation may comprise the auto-focus subsystem automatically focusing on the content within the ROI of the display FOV);
in response to an activation criteria being met, enter a focus mode that reduces distraction in the extended reality environment ([0080] detecting a direction of gaze of each eye of a user or a direction or location of focus); and
while in the focus mode: receive imaging data of the physical environment around the user using the first set of sensors ([0041] composite rendering is substantially similar to what the local user views through the see-through mixed-reality display on the local HMD device);
Swaminathan discloses identify, using the imaging data, a target object that is associated with the selected distraction type and is positioned within a second region of the physical environment [0040] presentation region for a media content item may be deployed to an augmented reality environment by a user for the purpose of defining a location (e.g., a position and/or orientation) where the media content item is presented within that augmented reality environment),
and generate the extended reality environment that includes a second reproduction of the second region of the physical environment with the target object (0003] Such supplemental information may form an information layer that overlays real-world objects that are captured in a camera view of the mobile device [0040] presentation region for a media content item may be deployed to an augmented reality environment by a user for the purpose of defining a location (e.g., a position and/or orientation) where the media content item is presented within that augmented reality environment, [0061] the AR information can be an AR object tag which represents additional data displayed over the image. [0078] the object tag 504-1 can be hidden (i.e., a visible property can be set to false). The transition between the show and hide state of the object tag can be a fade, dissolve, swipe, or other graphical transition).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include identify, using the imaging data, a target object that is associated with the selected distraction type and is positioned within a second region of the physical environment; and generate the extended reality environment that includes a second reproduction of the second region of the physical environment with the target object replaced with additional content as described by Swaminathan.
The motivation for doing so would have been for displaying an image to a user of a scene viewable by the user, receiving information indicative of an eye gaze of the user, determining an area of interest within the image based on the eye gaze information, causing an object tag to be displayed in an image of the scene based on the location of the area of interest, and displaying augmented reality information if the area of interest lingers on or about the object tag (Swaminathan, [0005]).
Mei discloses identify, using the imaging data, a target object that is associated with the selected distraction type and is positioned within a second region of the physical environment ([0032] the system determines whether the patient's gaze has a correlation with the primary object 210 above a threshold attention criterion);
wherein the reproduction includes a subregion in which the imaging data associated with the identified object is removed from the extended reality environment ([0031] when a threshold distraction criterion is exceeded, the system removes the correlated secondary object from the displayed scene)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include identify, using the imaging data, a target object that is associated with the selected distraction type and is positioned within a second region of the physical environment, wherein the reproduction includes a subregion in which the imaging data associated with the identified object is removed from the extended reality environment as described by Mei.
The motivation for doing so would have been to reduce the complexity of the displayed scene. (Mei, [0004]).
Lei, Swaminathan and Mei are combinable because they are from the same field of invention.
Therefore, it would have been obvious to combine Lei, Swaminathan and Mei to obtain the invention as specified in claim 10.
Regarding claim 11, Lei discloses detecting the user engagement with the candidate object comprises determining an amount of time that a gaze position of the user corresponds to the candidate object ([0054] Auto-focus operations may be disabled when one or more hands are not statically located for a duration of time that satisfies a pre-set threshold limit (e.g., 3 seconds)).; and
the selection criteria is a time threshold ([0054] pre-set threshold limit).
Regarding claim 12, Lei discloses detect the user engagement with the candidate object while in the focus mode ([0049] auto-focus subsystem may be configured with one or more criteria that, upon satisfaction or dissatisfaction, are determinative as to whether the HMD device triggers or suppresses an auto-focus operation.).
Regarding claim 13, Lei discloses the target object is the candidate object ([0027] virtual objects in the FOV coexist with real objects in the three-dimensional (3D) physical environment).
Regarding claim 14, Lei is silent to wherein the additional content is selected based on the distraction type.
Swaminathan discloses wherein the additional content is selected based on the distraction type ([0039] Augmented reality information may be determined, for example, by application of active segmentation and object recognition techniques to obtain electronic representations of the target areas and/or objects)
Lei, Swaminathan and Mei are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include wherein the additional content is selected based on the distraction type as described by Swaminathan.
The motivation for doing so would have been for displaying an image to a user of a scene viewable by the user, receiving information indicative of an eye gaze of the user, determining an area of interest within the image based on the eye gaze information, causing an object tag to be displayed in an image of the scene based on the location of the area of interest, and displaying augmented reality information if the area of interest lingers on or about the object tag (Swaminathan, [0005]).
Therefore, it would have been obvious to combine Lei, Swaminathan and Mei to obtain the invention as specified in claim 14.
Regarding claim 15, Lei discloses determine a current activity of the user ([0045] ROI may be located at a central region of the display system FOV as that is a likely location at which the user gazes);
determine whether the target object is associated with the current activity ([0045] During interactions with holograms of the virtual objects, the hands of the user may be located within the ROI which is defined by a set of spherical coordinates.); and
generate the extended reality environment that includes the second reproduction in response to determining that the target object is not associated with the current activity ([0046] auto-focus subsystem collectively utilizes this data to control the auto-focus operations to reduce the auto-focus hunting effect created by hand movements during interactions with the rendered holographic virtual objects.).
Regarding claim 16, Lei discloses prompt the user to confirm the selected distraction type ([0048] HMD device may be configured to detect and recognize hands, however, to enable gesture inputs and to further influence the auto-focus operations).
Regarding claim 17, Lei discloses a method for operating a head-mounted device during a focus mode that reduces distraction in an extended reality environment ([0027] mixed-reality environment supported on an HMD device), the method comprising:
receiving imaging data of a physical environment around a user using a set of sensors ([0034] HMD device can include many types of sensors);
identifying, using the imaging data, a candidate object in a portion of the physical environment ([0080] a gaze detection subsystem configured for detecting a direction of gaze of each eye of a user or a direction or location of focus);
selecting a first distraction criteria ([0046] auto-focus subsystem collectively utilizes this data to control the auto-focus operations to reduce the auto-focus hunting effect created by hand movements during interactions with the rendered holographic virtual objects.).;
displaying the extended reality environment using a display ([0039] holographic virtual objects that are viewable by the user through the see-through mixed-reality display system on the HMD device);
selecting a second distraction criteria ([0054] Auto-focus operations may be disabled when one or more hands are not statically located for a duration of time that satisfies a pre-set threshold limit (e.g., 3 seconds));
displaying the updated extended reality environment with the display ([0039] physical environment in which the user interacts with holographic virtual objects that are viewable by the user through the see-through mixed-reality display system on the HMD device).
Swaminathan discloses determining that the candidate object does not meet the first distraction criteria ([0064] determining that the user is not looking at the display);
generating the extended reality environment that includes a first reproduction of the portion of the physical environment with the candidate object ([0040] presentation region for a media content item may be deployed to an augmented reality environment by a user for the purpose of defining a location (e.g., a position and/or orientation) where the media content item is presented within that augmented reality environment).
determining that the candidate object meets the second distraction criteria ([0040] transition can include stages based on the time the user's eye gaze lingers on the activation region.)
updating the extended reality environment to include a second reproduction of the portion of the physical environment that (0003] Such supplemental information may form an information layer that overlays real-world objects that are captured in a camera view of the mobile device [0040] presentation region for a media content item may be deployed to an augmented reality environment by a user for the purpose of defining a location (e.g., a position and/or orientation) where the media content item is presented within that augmented reality environment, [0061] the AR information can be an AR object tag which represents additional data displayed over the image, [0078] the object tag 504-1 can be hidden (i.e., a visible property can be set to false). The transition between the show and hide state of the object tag can be a fade, dissolve, swipe, or other graphical transition.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include determining that the candidate object does not meet the first distraction criteria; generating the extended reality environment that includes a first reproduction of the portion of the physical environment with the candidate object; determining that the candidate object meets the second distraction criteria; updating the extended reality environment to include a second reproduction of the portion of the physical environment that replaces the candidate object with additional content as described by Swaminathan.
The motivation for doing so would have been for displaying an image to a user of a scene viewable by the user, receiving information indicative of an eye gaze of the user, determining an area of interest within the image based on the eye gaze information, causing an object tag to be displayed in an image of the scene based on the location of the area of interest, and displaying augmented reality information if the area of interest lingers on or about the object tag (Swaminathan, [0005]).
Mei discloses determining that the candidate object meets the second distraction criteria ([0032] the system determines whether the patient's gaze has a correlation with the primary object 210 above a threshold attention criterion);
updating the extended reality environment to include a second reproduction of the portion of the physical environment that removed the candidate object from the extended reality environment removed the candidate object from the extended reality environment ([0031] when a threshold distraction criterion is exceeded, the system removes the correlated secondary object from the displayed scene)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify mixed-reality environment of Lei to include determining that the candidate object meets the second distraction criteria, updating the extended reality environment to include a second reproduction of the portion of the physical environment that removed the candidate object from the extended reality environment removed the candidate object from the extended reality environment as described by Mei.
The motivation for doing so would have been to reduce the complexity of the displayed scene. (Mei, [0004]).
Lei, Swaminathan and Mei are combinable because they are from the same field of invention.
Therefore, it would have been obvious to combine Lei, Swaminathan and Mei to obtain the invention as specified in claim 17.
Regarding claim 18, Lei discloses determining that a current activity of the user is a first activity ([0045] ROI may be located at a central region of the display system FOV as that is a likely location at which the user gazes); and
determining that the current activity has changed from the first activity to a second activity ([0049] determinative as to whether the HMD device triggers or suppresses an auto-focus operation.), wherein:
the first distraction criteria is selected using the first activity ([0046] auto-focus subsystem collectively utilizes this data to control the auto-focus operations to reduce the auto-focus hunting effect created by hand movements during interactions with the rendered holographic virtual objects.); and
the second distraction criteria is selected using the second activity ([0050] user's hands are used in passing to control the hologram but the hologram is still the user's main point of interest.).
Regarding claim 19, Lei discloses the additional content is selected using the second activity ([0071] set focus in camera in a mixed-reality environment with hand gesture interaction)
Regarding claim 20, Lei discloses determining that an attentive state of the user is a first state ([0054] Auto-focus operations may be disabled when one or more hands are not statically located for a duration of time that satisfies a pre-set threshold limit (e.g., 3 seconds)).; and
determining that the attentive state has changed from the first state to a second state ([0084] gaze detection subsystem, also may be employed as user input devices, such that a user may interact with the display system via gestures of the eye,), wherein:
the first distraction criteria is selected using the first state ([0046] auto-focus subsystem collectively utilizes this data to control the auto-focus operations to reduce the auto-focus hunting effect created by hand movements during interactions with the rendered holographic virtual objects.); and
the second distraction criteria is selected using the second state ([0050] user's hands are used in passing to control the hologram but the hologram is still the user's main point of interest.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIVANG I PATEL whose telephone number is (571)272-8964. The examiner can normally be reached on M-F 9-5am.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached on (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIVANG I PATEL/Primary Examiner, Art Unit 2615
Prosecution Insights