CTNF 18/940,168 CTNF 88135 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Information Disclosure Statement The information disclosure statement (IDS) submitted on 11/7/24 is being considered by the examiner. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-3 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Stall et al. (US 2018/0330515 A1) . As to Claim 1 , Stall teaches A method of operating an electronic device, comprising: with one or more image sensors within a secure domain, acquiring an image (Stall discloses “a "video see-through" mode in which the headset 104 captures an image of the physical environment of the user” in [0025]) ; with a first subsystem within a general purpose domain separate from the secure domain, rendering one or more content layers (Stall discloses “by parallelizing the processing workload of the rendering of the virtual environment and the application view” in [0010]) ; with a second subsystem within the secure domain, processing the acquired image to produce a processed image without conveying the acquired image to the general purpose domain (Stall discloses “In some embodiments, when viewed from a particular perspective (e.g., when viewed from an angle), an object within the application may extend beyond the visual confines of the application region. In other embodiments, the object may be clipped to and confined within the application region” in [0035]; “As a third example 716, the application region 202 may be clipped, such that when a portion of the application view 206 extends beyond a boundary of the application region 202 from the perspective 106 of the user 102, the virtual environment 106 may clip the application view 206 to the boundary of the application region 202 from the perspective 106 of the user 102” in [0069]; applicant environment and virtual environment is mutually exclusive.) ; and with a third subsystem within the secure domain, combining the processed image with the one or more content layers (Stall discloses “The virtual environment 106 receives, from the application 114, an application view 206 of the application from the perspective 108 of the user 102, and inserts the application view 206 of the application 114 into the application region 202 of the virtual environment 106” in [0042], see also virtual environment compositor system 308 in [0050].) Stall doesn’t explicitly use claim language “secure domain” and “a general purpose domain”. However, Stall teaches a virtual environment and application environment are mutually exclusive in [0003, 0030-0031]; secret or proprietary techniques in [0008]. Here, application environment can be interpreted as a secure domain, while the virtual environment can be a general purpose domain. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Stall with the setting of different security levels of the rendering in the different regions within a virtual environment. As to Claim 2 , the modified Stall teaches The method of claim 1, wherein the first subsystem comprises a graphics rendering engine, and wherein rendering the one or more content layers comprises: with the graphics rendering engine, rendering a virtual content layer (Stall discloses “and by utilizing the existing rendering process of the application 114, such as shaders, lighting models, and rendering techniques…” in [0045].) As to Claim 3 , the modified Stall teaches The method of claim 2, wherein the virtual content layer comprises one or more user interface elements (Stall discloses “These techniques may enable the virtual environment to include the graphical user interfaces of one or more applications in a manner that is well-integrated with the virtual environment…” in [0010], see also [0025, 0027].) As to Claim 8 , the modified Stall teaches The method of claim 1, further comprising: with a rendering management subsystem within the general purpose domain, controlling blending operations at the first subsystem within the general purpose domain and blending operations at the second subsystem within the secure domain (Stall discloses “In this first example scenario 1000, a virtual environment compositor system 308 and an application 114 interoperate to generate two surfaces that respectively incorporate the output of the application 114, but do so in a manner that is not necessarily used for stereoscopic presentation; rather, the surfaces 208 present different perspectives 108 of the content of the virtual environment, including the application 114, at different times, such as views of a scene at a first time 214 and a second time 216” in [0092], see also Fig 10-11.) 07-21-aia AIA Claim s 4-7, 9-10 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Stall in view of Hawker et al. (US 2023/0230295 A1) As to Claim 4 , the modified Stall teaches The method of claim 2, wherein rendering the one or more content layers further comprises: with the graphics rendering engine, rendering a camera mask (Stall discloses “The virtual environment compositor system 308 also comprises a virtual environment presenter 312, which defines the application region 202 within the virtual environment 106 and identifies the perspective 108 of the user 102 within the virtual environment 106. The virtual environment presenter 312 also inserts the application view 206 of the application 114 , received by the application interface 310, into the application region 202 of the virtual environment 116, and presents the virtual environment 106, including the application environment 116 within the application region 202, to the user 102” in [0051]; “In this example scenario 200, the application region 116 again comprises a set of application models 118, such as text, images, movies, and/or two- and/or three-dimensional models that comprise the application environment 116” in [0037]. Here, it is well-known that the portion of virtual environment is masked by application region to display application view (i.e. images or movies etc.). For example, Hawker discloses “FIG. 2B is a flowchart depicting a method for generating a mask for a graphical element based on an outline of the graphical element” in [0032].) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Stall with the teaching of Hawker so as to generate a mask based on shape of graphical element. As to Claim 5 , Stall in view of Hawker teaches The method of claim 4, wherein the second subsystem comprises an image transform subsystem, and wherein processing the acquired image to produce the processed image comprises transforming the acquired image based on perspective information (Stall discloses “This feature may enable the user 102 to view and/or hide application models 118 or portions thereof at the boundaries of the application region 202 by shifting perspective 108 to make such application models 118 appear and/or disappear, respectively” in [0043]; “the virtual environment 106 may rotate the application region 206 to face the perspective 108 of the user 102. Rotating the application region 206 may enable the virtual environment 106 to maintain the stereoscopic aspect of the application view 206” in [0072].) As to Claim 6 , Stall in view of Hawker teaches The method of claim 5, wherein the third subsystem comprises a compositor subsystem, and wherein combining the processed image with the one or more content layers comprises: with the compositor subsystem, computing a product of the processed image and the camera mask (Stall discloses “In this first example 124, the application 114 comprises a three-dimensional application environment, such as a three-dimensional rendering of a scene, that is to be integrated with the virtual environment 106” in [0028], see also [0037].) As to Claim 7 , Stall in view of Hawker teaches The method of claim 6, wherein combining the processed image with the one or more content layers further comprises: with the compositor subsystem, computing a sum of the product and the virtual content layer (Stall discloses a virtual environment compositor system 308 in [0051], see also [0091-0092].) As to Claim 9 , Stall teaches A method of operating an electronic device, comprising: with one or more cameras, acquiring an image having camera pixel values (Stall discloses “a "video see-through" mode in which the headset 104 captures an image of the physical environment of the user” in [0025]; left/right camera in [0082]) ; and with a compositor, masking the image with the camera mask (Stall discloses “In this first example 124, the application 114 comprises a three-dimensional application environment, such as a three-dimensional rendering of a scene, that is to be integrated with the virtual environment 106” in [0028], see also [0037].) Stall doesn’t explicitly teach calculating alpha value. The combination of Hawker further teaches following limitations: with a graphics rendering engine, rendering a camera mask having alpha values that are independent of the camera pixel values (Stall discloses “The virtual environment compositor system 308 also comprises a virtual environment presenter 312, which defines the application region 202 within the virtual environment 106 and identifies the perspective 108 of the user 102 within the virtual environment 106. The virtual environment presenter 312 also inserts the application view 206 of the application 114 , received by the application interface 310, into the application region 202 of the virtual environment 116, and presents the virtual environment 106, including the application environment 116 within the application region 202, to the user 102” in [0051]; “As another example, the headset 104 may blend the surfaces 208, e.g., merging the surfaces 208 with a 50%/50% alpha channel to present a graphic composite of the scene at different times” in [0092]; “e.g., some calculations of the application 114, such as a perspective-dependent level of transparency of a semi-transparent and surface, may be materially identical for both perspectives 208, and a single evaluation by the application 114 of the transparency for a single frame may be applicable to the generation of the surfaces 1102 for both users 102” in [0093]. Here, it is well-known that the portion of virtual environment is masked by application region to display application view (i.e. images or movies etc. For example, Hawker discloses “For example, generating a mask shape for a graphical element having an opacity of 75% would result in a mask shape having an opacity of 25%... In another example, the alpha of the mask shape (maskAlpha) may be calculated from the alpha of the graphical element (sourceAlpha) in accordance with: maskAlpha = 100% - (sourceAlpha ×(100%/threshold))” in [0042].) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Stall with the teaching of Hawker so that a mask can be generated based on an inverted opacity of the graphical element (Hawker, [0042]). As to Claim 10 , Stall in view of Hawker teaches The method of claim 9. The combination of Hawker further teaches obtaining the alpha values based on a shape of the camera mask (Hawker discloses “FIG. 2B is a flowchart depicting a method for generating a mask for a graphical element based on an outline of the graphical element” in [0032]; “where the opacity of the layer at any given location in the layer may be described in an alpha channel or opacity map… For example, generating a mask shape for a graphical element having an opacity of 75% would result in a mask shape having an opacity of 25%” in [0042].) As to Claim 12 , Stall in view of Hawker teaches The method of claim 10, further comprising: before masking the image with the camera mask, transforming the image based on the shape of the camera mask (Stall discloses “In other embodiments, the object may be clipped to and confined within the application region” in [0035], see also [0043].) As to Claim 13 , Stall in view of Hawker teaches The method of claim 9, further comprising: with the graphics rendering engine, rendering a virtual content layer; and with the compositor, adding the virtual content layer to the masked image (Stall discloses to define an application region 202 within the virtual environment 106 and insert the application view 206 of the application 114 into the application region in [0051]; “a user may watch an overlay of a scene at different times to perform a visual comparison and identify material differences in the content of the application 114 at different times” in [0092].) As to Claim 14 , Stall in view of Hawker teaches The method of claim 9, wherein: the graphics rendering engine is part of a general purpose domain; the one or more cameras and the compositor are part of a secure domain that is separate from the general purpose domain; and the camera pixel values of the acquired image remain entirely within the secure domain and are isolated from the general purpose domain (Stall teaches a virtual environment and application environment are mutually exclusive in [0003, 0030-0031]; “where the virtual environment utilizes a different rendering process than the application environment; e.g., the application environment may utilize a particular selection of lighting models, shaders, geometry, and/or textures, including the platforms that implement them; however, the device upon which the holographic view is executing may provide no such support for secret or proprietary techniques” in [0008]. Here, application environment can be interpreted as a secure domain, while the virtual environment can be a general purpose domain. Each domain can have different set of engines. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Stall with the setting of different security levels of the rendering in the different regions within a virtual environment.) 07-21-aia AIA Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Stall in view of Hawker and Chiu et al. (US 2023/0316634 A1) . As to Claim 11 , Stall in view of Hawker teaches The method of claim 10, wherein the alpha values comprise gradually increasing or decreasing alpha values along an edge of the shape of the camera mask (Chiu discloses “computer system 101 displays a gradual increase and/or decrease of opacity starting from one or more inner edges of effect region 1906 relative to a center of object 1914a” in [0461].) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Stall and Hawker with the teaching of Chiu so as to display a gradual transition in visual prominence of object by gradually increase or decrease opacity along an edge of the region (Chiu, [0461]) . 07-21-aia AIA Claim s 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Stall in view of Hawker and Woodall (US 8,358,379 B1) . Claim 15 recites similar limitations as claim 9, besides black camera pixel value. It is well-known that on screen display (i.e. caption) may have black background. For example, Woodall discloses “on screen display data overlaid on a black background” in Abstract. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Stall and Hawker with the teaching of Woodall so as to apply an alpha blend value to the image data to produce processed imaged data and the on screen display data. As to Claim 16 , Stall in view of Hawker and Woodall teaches The method of claim 15, wherein rendering the virtual content layer comprises: blending the black camera pixel values with foreground pixel values based on alpha values that are independent of the camera pixel values (Woodall discloses “A method of processing on screen display data with an image post processor includes receiving a data stream from a video processor at a post processing device having at least one port, the data stream including on screen display data overlaid on a white background and the on screen display data overlaid on a black background, finding a difference between the on screen display data overlaid on white and the on screen display data overlaid on black, using the difference to determine a complement of an alpha blend value, performing image processing on the image data with the post processor by applying the complement of an alpha blend value to the image data to produce processed image data, and transmitting the processed image data and the on screen display data through a display port” in Abstract.) Claim 17 is rejected based upon similar rationale as Claim 9. Claim 18 is rejected based upon similar rationale as Claim 12. As to Claim 19 , Stall in view of Hawker and Woodall teaches The method of claim 17, wherein rendering the camera mask comprises rendering the camera mask based on camera pixel blending alpha values that are independent of the camera pixel values (Stall discloses “As another example, the headset 104 may blend the surfaces 208, e.g., merging the surfaces 208 with a 50%/50% alpha channel to present a graphic composite of the scene at different times” in [0092]; “e.g., some calculations of the application 114, such as a perspective-dependent level of transparency of a semi-transparent and surface, may be materially identical for both perspectives 208, and a single evaluation by the application 114 of the transparency for a single frame may be applicable to the generation of the surfaces 1102 for both users 102” in [0093].) As to Claim 20 , Stall in view of Hawker and Woodall teaches The method of claim 15, wherein: the graphics rendering engine comprises a subsystem within a general purpose domain; the one or more cameras comprises one or more sensors within a secure domain that is separate from the general purpose domain; the camera pixel values are isolated from the general purpose domain; and one or more processors in the secure domain are configured to produce the processed image and to produce the composite image (Stall teaches a virtual environment and application environment are mutually exclusive in [0003, 0030-0031]; “where the virtual environment utilizes a different rendering process than the application environment; e.g., the application environment may utilize a particular selection of lighting models, shaders, geometry, and/or textures, including the platforms that implement them; however, the device upon which the holographic view is executing may provide no such support for secret or proprietary techniques” in [0008]. Here, application environment can be interpreted as a secure domain, while the virtual environment can be a general purpose domain. Each domain can have different set of engines. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Stall with the setting of different security levels of the rendering in the different regions within a virtual environment.) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEIMING HE whose telephone number is (571)270-1221. The examiner can normally be reached on Monday-Friday, 8:30am-5:00pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached on 571-272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /WEIMING HE/ Primary Examiner, Art Unit 2611 Application/Control Number: 18/940,168 Page 2 Art Unit: 2611 Application/Control Number: 18/940,168 Page 3 Art Unit: 2611 Application/Control Number: 18/940,168 Page 4 Art Unit: 2611 Application/Control Number: 18/940,168 Page 5 Art Unit: 2611 Application/Control Number: 18/940,168 Page 6 Art Unit: 2611 Application/Control Number: 18/940,168 Page 7 Art Unit: 2611 Application/Control Number: 18/940,168 Page 8 Art Unit: 2611 Application/Control Number: 18/940,168 Page 9 Art Unit: 2611 Application/Control Number: 18/940,168 Page 10 Art Unit: 2611 Application/Control Number: 18/940,168 Page 11 Art Unit: 2611 Application/Control Number: 18/940,168 Page 12 Art Unit: 2611 Application/Control Number: 18/940,168 Page 13 Art Unit: 2611 Application/Control Number: 18/940,168 Page 14 Art Unit: 2611 Application/Control Number: 18/940,168 Page 15 Art Unit: 2611
Prosecution Insights