DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Amendment filed on 2/23/2026.
Claims 21-50 are pending. Claims 21-36, 38-40, 43, 46 have been amended.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/23/2026 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 21-22, 24-35, 36-37, 39-40, 42-50 is/are rejected under 35 U.S.C. 103 as being unpatentable over Haddick et al. (US 20120212484 A1, hereinafter Haddick), in view of Marks et al. (US 20170269713 A1, hereinafter Marks), further in view of Eluvan et al. (US 20230186528 A1, hereinafter Eluvan).
Regarding Claim 21, Haddick teaches a mixed reality wearable apparatus (Haddick, Paragraph [0006], [0858], “The present disclosure relates to an augmented reality eyepiece”; “control aspects of the eyepiece may include combinations of using user action capture inputs/ devices and command/control modes and interfaces in which the inputs can be reflected, such as wearable sensor sets”), comprising: at least one camera (Haddick, Paragraph [0332], “The front portion of the frame may also be used to mount a camera or image sensor”);
at least one memory storing instructions (Haddick, Paragraph [0334], “The DSP may include a memory…for storing static information and instructions for the internal processor”); and at least one hardware processor configured to execute the instructions to perform operations comprising: receiving a live video stream of a physical environment from the at least one camera (Haddick, Paragraph [0334], “a digital signal processor (DSP) may be programmed and/or configured to receive video feed information and configure the video feed to drive whatever type of image source is being used with the optical display”; [0483], “the wearer may enter into a videoconference with a plurality of other people, where the wearer may be able to view live video images” [0856], “may be constantly viewing the surrounding environment though the eyepiece” [0967], “allow the user to share live video”); generating an internal view including at least a first portion and a second portion of the live video stream of the physical environment and a virtual reality object (Haddick, Paragraph [0483], “the wearer may enter into a videoconference with a plurality of other people, where the wearer may be able to view live video images”; [0856], “is continuously streamed a visual display overlay that allows for the soldier to have an augmented reality view of the surrounding environment”; [0226], “operated to provide sequential portions <read on first portion and second portion> of the image to the user”); generating from the internal view an external view including the first portion of the live video stream of the physical environment and the virtual reality object (Haddick, Paragraph [1021], “head mounted displays <read on internal view> are used to add images of virtual objects or virtual information that is associated with the view of a scene <read on first portion of the live video stream> as seen by a user” [0283], “The projection lens 9350 is typically configured to project the modulated optical beam 9340 onto a display, such as a video display screen <read on external view>”) and [[ replacing the second portion of the live video stream of the physical environment with content alternative to the second portion of the live video stream of the physical environment ]]; the external view not including the second portion of the live video stream of the physical environment (Haddick, Paragraph [0362], there may be a plurality of chromic areas on the lens that may be controlled independently, such as large portions of the lens, sub-portions of the projected area, programmable areas of the lens and/or projected area… enabled with certain applications (e.g. a streaming video application, a sun tracking application, an ambient brightness sensor; [0637], “data may be filed using the most salient image plus manual entry, enabling partial data capture”); [[ and at least thereby being different from the internal view; ]]
But Haddick does not explicitly disclose [[ the external view not including the portion of the video stream ]] and at least thereby being different from the internal view; simultaneously displaying the internal view at the mixed reality wearable apparatus and casting the external view, different from the internal view, from the mixed reality wearable apparatus to an external display device separate from the mixed reality wearable apparatus.
However, Marks teaches the external view not including the second portion of the live video stream of the physical environment and at least thereby being different from the internal view (Marks, Paragraph [0011], [0020], [0056], [0011], “The feed of the virtual reality stream provides an HMD spectator view into the virtual reality environment that is enabled to be a same view and distinct views relative to the HMD player view”; [0020], “The viewing spot is decoupled from the HMD view” [0056], “provide spectators with decoupled views from the view of the HMD player”); simultaneously displaying the internal view at the mixed reality wearable apparatus and casting the external view, different from the internal view, from the mixed reality wearable apparatus to an external display device separate from the mixed reality wearable apparatus. (Marks, Paragraph [0030], “The feed of the virtual reality environment is shared to a website… Then, sending, by the server, a spectator view into the virtual reality environment to a device of the spectator. [0034], “an HMD used by the spectator or a television screen used by the spectator, or a computer screen used by the spectator, or hand-held device screen used by the spectator” [0083], “The client system 106 can, in one embodiment receive the second screen content from one of the content sources 120, or from a local user, or a remote user” “the client system 106 can be used to simultaneously process content for an HMD user, such that the HMD is provided with multimedia content associated with the interactivity during gameplay”).
Marks and Haddick are analogous since both concern head-mounted display (HMD) systems that present an immersive user view while enabling spectators to observe the experience on external screens or websites. Haddick provided a way of capturing camera imagery and rendering an AR-based internal view to the wearer through a near-eye display, with communication links to exchange video or data with remote systems. Marks provided a way of creating a spectator-view pipeline that is technically decoupled from the wearer's head-tracked view using a virtual camera then sending that spectator feed to external devices or websites. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate spectator-view generation and distribution framework taught by Marks into modified invention of Haddick such that system will be able to allowing viewers on external displays to follow or comment in real time, improving collaboration, demonstrations, live broadcasting, and social interaction while preserving the integrity of the wearer's immersive view.
The combination does not explicitly disclose but Eluvan teaches replacing the second portion of the live video stream of the physical environment with content alternative to the second portion of the live video stream of the physical environment (Eluvan, Paragraph [0072], “the actual background behind the third viewing user 736 is replaced with the AR background 738 such that the third viewing user 736 appears to be sitting in the crowd”; [0057], “The enhanced version of the live video feed includes the original version of the live video feed supplemented by the AR overlay content generated at task 510. In this regard, AR overlay content may be superimposed over a portion of the live video feed, added to the live video feed, or combined with the live video feed. In some scenarios, AR overlay content can be incorporated into the live video feed such that it replaces some of the original content. For example, the background behind the viewing user's body can be replaced with or covered by AR overlay content”);
Eluvan and Haddick are analogous since both concern camera-equipped computing systems that capture a live video feed of a physical environment, apply augmented reality content as an overlay to produce an enhanced view, and present that enhanced view to users or external audiences. Haddick provides a way of capturing live camera imagery of a physical environment through a wearable eyepiece and rendering an AR based internal view to the wearer by overlaying virtual objects onto the live physical scene, with the resulting view available for display or communication to external systems. Eluvan provides a way of processing a live camera video feed of a physical environment and selectively replacing a spatial portion of that live feed – specifically the background region - with AR alternative content, while preserving another portion of the live feed , thereby generating an enhanced version of the live video feed that differs from the original captured view. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the selective spatial portion replacement technique taught by Eluvan into the modified invention of Haddick, such that the system would be able to replace a designated region of the live physical environment video stream with alternative AR content when generating an external view, improving audience experience, protecting user privacy, enabling contextual customization of what external viewers observe, and allowing content creators to control the presentation of the physical environment in the cast view while preserving the integrity of other portions of the live video stream.
Regarding Claim 22, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the external view is generated based on a user interaction with a button, control, icon, or toggle (Huddick, Paragraph [0025], “Command and control modes operating in conjunction with the eyepiece may be initiated by sensing inputs through input devices, user action, external device interaction, reception of events and/or data feeds, internal application execution, external application execution, and the like”).
Regarding Claim 24, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the content alternative to the second portion of the live video stream of the physical environment comprises a computer-generated rendering (Haddick, Paragraph [0226], “operated to provide sequential portions <read on first portion and second portion> of the image to the user” ; [0518],the eyepiece may provide 3D display imaging to the user, such as through conveying a stereoscopic, auto-stereoscopic, computer-generated holography, volumetric display image, stereograms/stereoscopes, view-sequential displays, electro-holographic displays, parallax "two view" displays [0522], eyepiece facilities may use visual techniques to render the projection of an object).
Regarding Claim 25, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the content alternative to the second portion of the live video stream of the physical environment comprises an image (Haddick, Paragraph [0226], “operated to provide sequential portions <read on first portion and second portion> of the image to the user”; [0399], “This technique is applicable to video cameras, shifting the electronic image from frame to frame of the video in a manner sufficient to counteract the motion”) .
Regarding Claim 26, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches the operations further comprising
presenting an interface configured to allow user selection of an external view mode (Haddick, Paragraph [0551], “The user selects by making a gesture using a pointing device, or gesture to signal the selection of the video call option”) in
which the external view includes a [[ solid ]] background (Haddick, Paragraph
[1026], In the background of FIG. 114, the 3D labels from FIG. 113 are coincident with and positioned on top of the 3D labels from FIG. 112 with no disparity) or a computer-generated rendering (Haddick, Paragraph [0518],the eyepiece may provide 3D display imaging to the user, such as through conveying a stereoscopic, auto-stereoscopic, computer-generated holography, volumetric display image, stereograms/stereoscopes, view-sequential displays, electro-holographic displays, parallax "two view" displays [0522], eyepiece facilities may use visual techniques to render the projection of an object).
Regarding Claim 27, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the external display device comprises a computer or a projector (Haddick, Paragraph [0816], “the eyepiece may provide control through facilities internal and external to the eyepiece, such as initiated from the surrounding environment 7202, input devices 7204, sensing devices 7208, user action capture devices 7210, internal processing facilities 7212, internal multimedia processing facilities, internal applications 7214, camera 7218, sensors 7220, earpiece 7222, projector 7224).
Regarding Claim 28, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein casting the external view comprises wirelessly transmitting the external view from the mixed reality wearable apparatus to the external display device (Haddick, Paragraph [0006], “The present disclosure relates to an augmented reality eyepiece”; [0561], “The eyepiece <read on wearable apparatus> may also use multiple wireless communications systems, such as one for streaming high data rate media (e.g. video), one for low data rate media (e.g. text messaging), one for command data between the external facility and the eyepiece).
Regarding Claim 29, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein casting the external view comprises casting the external view using a Local Area Network (LAN) or Bluetooth (Haddick, Paragraph [0986], “The internal application 7214 may interact with the eyepiece control process facility 10634 processes, such as in conjunction with an API 10608, through input devices 7204, external devices 7240, external computing facilities 7232, command and control 10630 facilities of the eyepiece, and the like. Internal applications 7214 may be made available to the eyepiece 100 through a network communications connection 10622, such as the Internet, a local area network (LAN)”;[0561], “the eyepiece may provide for an interface to accept wireless streaming media (e.g. video, audio, text messaging, phone call and calendar alerts) from an external facility…The wireless streaming media may be through any of the wireless communication systems and protocols know in the art, such as Bluetooth).
Regarding Claim 30, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the internal view and the external view are generated simultaneously (Marks, Paragraph [0083], “the client system 106 can be used to simultaneously process content for an HMD user, such that the HMD is provided with multimedia content associated with the interactivity during gameplay. The client system 106 can then also provide other content… to the second screen” [0030], “the method includes receiving, by the server, a feed of a virtual reality environment rendered for a head mounted display (HMD) of an HMD player. The HMD player is provided with an HMD view… The feed of the virtual reality environment is shared to a website. Then, sending, by the server, a spectator view into the virtual reality environment to a device of the spectator” [0106], “spectator can be local to the HMD player, and can be viewing the HMD content on a second screen… viewing from a webpage if the HMD player content is being published to a website that allows for viewing”).
Marks and Haddick are analogous since both relate to interactive mixed-reality environments that coordinate HMD content and external display experiences. Haddick provided a way of producing an internal HMO view for the wearer and communicating multimedia content over wired or wireless links. Marks provided a way of generating a spectator-oriented view that is technically decoupled/separated from the wearer’s head-tracked perspective by using an independent virtual-camera pipeline and transmitting the feed to other devices. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate concurrent second-screen streaming architecture taught by Marks into modified invention of Haddick such that system will be able to enable the HMD wearer's live content to be processed and mirrored to external displays without interruption, which facilitates real-time collaboration, broadcasting, and feedback between the participant inside the headset and observers outside it. This yields a more integrated, responsive XR ecosystem rather than a single-user closed display experience.
Regarding Claim 31, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein generating the internal view comprises overlaying the virtual reality object onto the first portion of the live video stream of the physical environment (Haddick, Paragraph [0226], “operated to provide sequential portions <read on first portion and second portion> of the image to the user” ; [0953], “The user may use the eyepiece to view <read on internal view > an overlay of visual and/or audio instructions of the item to walk the user through maintenance without the need for a handheld manual…video, still images, 3D and/or 2D images, animated images, audio and the like may be used for such maintenance”).
Regarding Claim 32, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the virtual reality object
comprises a three-dimensional object (Haddick, Paragraph [0953], “video, still images, 3D and/or 2D images, animated images, audio and the like may be used for such maintenance… the user may view an overlay and/or video of various images of the item”).
Regarding Claim 33, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the virtual reality object comprises a virtual addition to a physical object (Haddick, Paragraph [1010], “ergence works in conjunction with eye accommodation to permit a person to maintain a clear image of an object as the object moves relative to the person. Vergence compensation becomes important in situations where a virtual image, i.e., an AR image, such as a label or other information, is to be placed near to or overlap a real image or when a virtual image of an object is to be superimposed upon a real image of the object in order to make the placement of the virtual image correct with respect to the real image”).
Regarding Claim 34, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the virtual reality object comprises two-dimensional content (Haddick, Paragraph [1121], a method for positioning 2D labels in a 3D virtual world…method for augmented reality is to adjust the focus of the virtual objects or virtual information so that the user perceives differences in focus depth that provide a depth cue to the user).
Regarding Claim 35, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination further teaches wherein the virtual reality object comprises animate virtual content (Haddick, Paragraph [1016], “the user may be requested to look at a series of real or virtual objects having a range of real or virtual distances, e.g., from near to far, and the vergence of the eyes is measured either mechanically or electronically or both” [1021], method for augmented reality is to adjust the focus of the virtual objects or virtual information so that the user perceives differences in focus depth that provide a depth cue to the user. [0529], The sign may be a billboard, and the advertisement for a personalized advertisement based on a preferences profile of the user. The visual cue 3802,3808 may be a hand gesture, and the projected content a projected virtual keyboard [0972], “such images may be still and/or video, animated, 3-D, 2-D, and the like” [0591], the local advertisement may be animated in embodiments).
Regarding Claim 36, it recites limitations similar in scope to the limitations of claim 21 and the combination of Haddick, Marks and Eluvan teaches all the limitations as of Claim 1. And Haddick discloses these features can be implemented on a computer readable storage medium (Haddick, Fig. 29, Paragraph [0335], The DSP may include at least one computer readable medium or memory for holding instructions programmed and for containing data structures…Various forms of computer readable media may be involved in carrying out one or more sequences of one or more instructions to the optical display for execution).
Regarding Claim 37, it recites limitations similar in scope to the limitations of Claim 22 and therefore is rejected under the same rationale.
Regarding Claim 39, it recites limitations similar in scope to the limitations of Claim 24 and therefore is rejected under the same rationale.
Regarding Claim 40, it recites limitations similar in scope to the limitations of Claim 25 and therefore is rejected under the same rationale.
Regarding Claim 41, it recites limitations similar in scope to the limitations of Claim 26 and therefore is rejected under the same rationale.
Regarding Claim 42, it recites limitations similar in scope to the limitations of Claim 27 and therefore is rejected under the same rationale.
Regarding Claim 43, it recites limitations similar in scope to the limitations of Claim 28 and therefore is rejected under the same rationale.
Regarding Claim 44, it recites limitations similar in scope to the limitations of Claim 29 and therefore is rejected under the same rationale.
Regarding Claim 45, it recites limitations similar in scope to the limitations of Claim 30 and therefore is rejected under the same rationale.
Regarding Claim 46, it recites limitations similar in scope to the limitations of Claim 31 and therefore is rejected under the same rationale.
Regarding Claim 47, it recites limitations similar in scope to the limitations of Claim 32 and therefore is rejected under the same rationale.
Regarding Claim 48, it recites limitations similar in scope to the limitations of Claim 33 and therefore is rejected under the same rationale.
Regarding Claim 49, it recites limitations similar in scope to the limitations of Claim 34 and therefore is rejected under the same rationale.
Regarding Claim 50, it recites limitations similar in scope to the limitations of Claim 35 and therefore is rejected under the same rationale.
Claim(s) 23, 38 is/are rejected under 35 U.S.C. 103 as being unpatentable over Haddick et al. (US 20120212484 A1, hereinafter Haddick), in view of Marks et al. (US 20170269713 A1, hereinafter Marks), further in view of Eluvan et al. (US 20230186528 A1, hereinafter Eluvan) as applied to Claim 21, 36 above respectively and further in view of Krol et al. (US 11741652 B1, hereinafter Krol)
Regarding Claim 23, The combination of Haddick, Marks and Eluvan teaches the invention in Claim 21.
The combination does not explicitly disclose but Krol teaches wherein the content alternative to the second portion of the live video stream of the physical environment comprises a solid background (Krol, Column 8, Line 4-7, 12, Line 37-42, “Devices 306A-B can be any type of computing device, including a laptop, a desktop, a smartphone, a tablet computer, or a wearable computer (such as a smartwatch or a augmented reality or virtual reality headset)” “user 604 physically places a solid colored background behind them to assist in background alteration (see FIG. 6B for a general illustration). In this aspect, the video stream is analyzed to determine which portions represent user 604 and which represent the solid colored background; Column 2, Line 39-40, “the video on the avatar illustrates the other user's real world physical surroundings”).
Krol and Haddick are analogous since both of them are dealing with display media in the augmented reality environment. Haddick provided a way of allow user who wearing the eyepiece or glasses to see the object in the augmented reality and communicated to external device for sharing the data. Krol provided a way of allow user to select the solid color background of the live video stream when displaying and/or projecting data in collaboration augmented environment. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate solid background selection taught by Krol into modified invention off Haddick such that when viewing the medium in the augmented reality environment, system will be able to use solid background in the live video stream in order for users to properly and easily identify the object in the media they are looking. The motivation is to allow system is capable of automatically detecting the color of the physically placed background, so the analysis is conducted by detecting which pixels are that color (i.e., physically placed background) and which pixels are not that color (i.e., user 604) discussed by Krol in Column 16, Line 50-55
Regarding Claim 38, it recites limitations similar in scope to the limitations of Claim 23 and therefore is rejected under the same rationale.
Response to Arguments
Applicant’s arguments with respect to claim 21, 36, filed on 3/26/2026, with respect to rejection under 35 USC § 103 have been considered but are moot in view of the new ground(s) of rejection. it has now been taught by the combination of Haddick, Marks and Eluvan.
In regard to Claims 22-35, 37-50, they directly/indirectly depends on independent Claim 21, 36 respectively. Applicant does not argue anything other than the independent claim 21, 36. The limitations in those claims in conjunction with combination previously established as explained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20250169915 A1 System for providing remote collaborative treatment for tagging realtime surgical video
US 20250259399 A1 Augmented reality experience with occluder map prediction
US 20240152244 A1 Devices, Methods, and Graphical User Interfaces for Interacting with Three-Dimensional Environments
US 20250193344 A1 Generating a 3D Representation of a Head of a Participant in a Video Communication Session
US 20190356964 A1 Device for facilitating reality presentation associated with media program, has processing system for obtaining identification of participant in event and for presenting augmented reality marker through augmented reality device
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights