DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/07/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Claim Rejections - 35 USC § 103
1 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
2 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3 Claim(s) 1-5, 9-12, 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Border et al. (US 20160048021 A1) in view of Sugden et al. (US 20130141434 A1).
4 Regarding claim 1, Border teaches a method comprising: at an electronic device including one or more processors, , and a see-through display ([0100] reciting “The system may need to be designed as a lightweight, compact and fully functional computer display, such as wherein the computer display includes a high resolution digital display that provides a high level of emersion comprised of the displayed digital content and the see-through view of the environmental surroundings.”; [0104] reciting “The HWC 102 may also have a number of integrated computing facilities, such as an integrated processor, integrated power management, communication structures (e.g. cell net, WiFi, Bluetooth, local area connections, mesh connections, remote connections (e.g. client server, etc.)), and the like.”):
obtaining a light superposition characteristic value associated with ambient light from a physical environment that enters the see-through display ([0103] reciting “In situations where the HWC 102 has integrated computer displays the displays may be configured as see-through displays such that the digital imagery can be overlaid with respect to the user's view of the environment 114. There are a number of see-through optical designs that may be used, including ones that have a reflective display (e.g. LCoS, DLP), emissive displays (e.g. OLED, LED, backlit LCD), hologram, TIR waveguides, and the like.”; [0240] reciting “The brightness of the displayed image Bdi can then be adjusted in relation to Bstv by adjusting the brightness of the light source b or by scaling the code values in the image. The brightness of the see-through view Bstv can be changed by changing a portion of the see-through optics to change the tinted layer”);
obtaining a perceptual appearance value that characterizes the light superposition characteristic value ([0153] reciting “FIG. 13c illustrates an embodiment where the combiner element 1304 is angled away from the eye at the top and towards the eye at the bottom (e.g. in accordance with the holographic or notch filter embodiments described herein). In this embodiment, the effects LED 1308a is located on the outer lens 1302 side of the combiner element 1304 to provide a concealed appearance of the lighting effects.”; [0233] reciting “In some cases, t is broken into multiple subtimes within a frame time that together add to t to further reduce any perceptible flicker in the image.”);
determining a display correction value associated with the electronic device based on the light superposition characteristic value, a predetermined display characteristic of a computer-generated reality (CGR) object, and the perceptual appearance value ([0119] reciting “The embodiment of FIG. 4 also includes a corrective wedge 420 to correct the effect of refraction of the image light 414 as it exits the TIR wedge 418. By including the corrective wedge 420 and providing a thin air gap 408 (e.g. 25 micron), the image light from the “on” pixels can be maintained generally in a direction along the optical axis of the field lens (i.e. the same direction as that defined by the image light 414) so it passes into the field lens and the lower optical module 204.”; [0153] reciting “In this embodiment, the effects LED 1308a is located on the outer lens 1302 side of the combiner element 1304 to provide a concealed appearance of the lighting effects.”; [0275] reciting “FIG. 46 illustrates a see-through view as may be seen when using a HWC wherein information is overlaid onto a see-through view of the environment. The tree and the building are actually in the environment and the text is displayed in the see-through display such that it appears overlaid on the environment. In addition to text information such as, for example, instructions and weather information, some augmented reality information is shown that relates to nearby objects in the environment.”); and
changing one or more display operating parameters associated with the electronic device in accordance with the display correction value in order to satisfy the predetermined display characteristic of the CGR object within a performance threshold ([0100] reciting “The HWC may then change the mode of operation to match the conditions, location, positioning, movements, and the like, in a method generally referred to as a contextually aware HWC.”; [0119] reciting “In contrast, the dark state light 410 is substantially changed in direction by refraction when the dark state light 410 exits the corrective wedge 420.”; [0159] reciting “In an embodiment, the display(s) may change from color displays to monochrome displays to reduce the amount of light produced. In an embodiment, the monochrome lighting may be red to limit the impact on the wearer's eyes to maintain an ability to see better in the dark.”).
5 Although Border could teach a non-transitory memory ([0163] reciting “The commands may then be communicated back to the HWC 102 for execution (e.g. display writing in the glasses display, make a selection within the UI of the glasses display, control a remote external device 112, control a local external device 108), and the like. The pen may also include memory 1514 for long or short term uses.”), prior art from Sugden can further teach the limitations.
6 Sugden teaches a non-transitory memory ([Abstract] reciting “Graphical content presented via the see-through display is created by modeling the ambient lighting conditions of the physical environment.”; [0004] reciting “A graphical representation of the illuminated virtual environment is rendered for presentation via the see-through display of the head-mounted display system to thereby visually augment an appearance of the physical environment to a user viewing the physical environment through the see-through display.”; [0057] reciting “Data-holding subsystem 514 may include one or more physical, non-transitory, devices configured to hold data and/or instructions executable by the logic subsystem to implement the herein described methods and processes.”)…
7 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Border) to incorporate the teachings of Sugden to provide a type of non transitory memory for the ambient light methods that are taught by Border. Doing so would allow configuration to hold data and/or instructions executable by the logic subsystem as stated by Sugden ([0057] recited).
8 Regarding claim 2, Border in view of Sugden teaches the method of claim 1, wherein the perceptual appearance value (see claim 1 rejection above) is based on a color appearance model (Border; [0112] reciting “Full color images can be presented to a user by sequentially providing illumination light with complimentary colors such as red, green and blue. Where the sequence is presented in a recurring cycle that is faster than the user can perceive as separate images and as a result the user perceives a full color image comprised of the sum of the sequential images.”).
9 Regarding claim 3, Border in view of Sugden teaches the method of claim 1, wherein the perceptual appearance value (see claim 1 rejection above) is based on a user attribute (Border; [0319] reciting “The attribute may be something that more precisely places the content within the environment located at the geo-spatial location. The attribute may be selected such that content appears in a hallway, office, near a billboard, rooftop, outside wall, object, etc. Personal information relating to the person may be stored such that it can be retrieved during a process of determining at what physical location in the world certain digital content should be presented to the person.”; [0356] reciting “In embodiments, known objects in the environment proximate the user are recognized to assist with the distance determination. For example, a HWC 102 may have access to a database of objects (e.g. on-board, server based, etc.) and the camera on the HWC 102 may be activated to look for any appearance of the objects when it is desirable to display content that is intended to have the appearance that the content is proximate an object in the environment.”).
10 Regarding claim 4, Border in view of Sugden teaches the method of claim 3, wherein the user attribute (see claims 1 and 3 rejection above) is based on an eye gaze of a user (Border; [0293] reciting “For example, the garment may be of a certain style and the HWC may recognize the style and record it's viewing. The scene also includes a mapped object 5718 and a recognized object 5720. As the wearer moves through the scene, the sight and/or eye headings may be recorded and communicated from the HWC 102. In embodiments, the time that the sight and/or eye heading maintains a particular position may be recorded. For example, if a person appears to look at an object or person for a predetermined period of time (e.g. 2 seconds or longer), the information may be communicated as gaze persistence information as an indication that the person may have been interested in the object.”).
11 Regarding claim 5, Border in view of Sugden teaches the method of claim 1, wherein changing the one or more display operating parameters (see claim 1 rejection above) includes changing a tint level associated with the see-through display in order to satisfy the predetermined display characteristic of the CGR object within the performance threshold (Border; [0159] reciting “In an embodiment, the display(s) may change from color displays to monochrome displays to reduce the amount of light produced. In an embodiment, the monochrome lighting may be red to limit the impact on the wearer's eyes to maintain an ability to see better in the dark.;” [0239] reciting “The brightness of the see-through view Bstv can be changed by changing a portion of the see-through optics to change the tinted layer, allowing the photochromic layer to automatically change optical density due to changes in the brightness of the environment, or by adjusting the optical density of the adjustable optical density layer or a combination thereof as the brightness of the environment changes.”).
12 Regarding claim 9, Border in view of Sugden teaches the method of claim 1, wherein changing the one or more display operating parameters includes (see claim 1 rejection above) changing a color composition associated with the see-through display in order to satisfy the predetermined display characteristic of the CGR object within the performance threshold (Border; [0159] reciting “For example, the display(s) may go into a simple content delivery mode to restrict the amount of information displayed. This may be done to reduce the amount of light produced by the display(s). In an embodiment, the display(s) may change from color displays to monochrome displays to reduce the amount of light produced. In an embodiment, the monochrome lighting may be red to limit the impact on the wearer's eyes to maintain an ability to see better in the dark.”; [0176] reciting “Generally, in the present disclosure, instrument stroke parameter changes may be referred to as a change in line type, line weight, tip type, brush type, brush width, brush pressure, color, and other forms of writing, coloring, painting, and the like.”; [0240] reciting “The brightness of the see-through view Bstv can be changed by changing a portion of the see-through optics to change the tinted layer, allowing the photochromic layer to automatically change optical density due to changes in the brightness of the environment, or by adjusting the optical density of the adjustable optical density layer or a combination thereof as the brightness of the environment changes.”).
13 Regarding claim 10, Border in view of Sugden teaches the method of claim 9, wherein changing the color composition associated with the see-through display includes (see claims 1 and 9 rejections above) adding colors to the see-through display as a function of the ambient light from the physical environment (Border; [0112] reciting “Full color images can be presented to a user by sequentially providing illumination light with complimentary colors such as red, green and blue. Where the sequence is presented in a recurring cycle that is faster than the user can perceive as separate images and as a result the user perceives a full color image comprised of the sum of the sequential images. Bright pixels in the image are provided by pixels that remain in the “on” state for the entire time of the cycle, while dimmer pixels in the image are provided by pixels that switch between the “on” state and “off” state within the time of the cycle, or frame time when in a video sequence of images.”).
14 Regarding claim 11, Border in view of Sugden teaches the method of claim 1, wherein changing the one or more display operating parameters includes (see claim 1 rejection above) setting rendering parameters for the CGR object in order to satisfy the predetermined display characteristic of the CGR object within the performance threshold (Border; [0159] reciting “In an embodiment, the display(s) may change from color displays to monochrome displays to reduce the amount of light produced. In an embodiment, the monochrome lighting may be red to limit the impact on the wearer's eyes to maintain an ability to see better in the dark.”; [0176] reciting “Generally, in the present disclosure, instrument stroke parameter changes may be referred to as a change in line type, line weight, tip type, brush type, brush width, brush pressure, color, and other forms of writing, coloring, painting, and the like.”; [0265] reciting “The content position may shift outside of the field of view. In embodiments, the content is no longer displayed if the speed of movement exceeds a predetermined threshold and will be displayed again once the forward motion slows.”; [0313] reciting “For example, the tactical plan may assist in the prediction of the location of the other BlueForce member and the virtual target line may be adjusted accordingly. In embodiments, the area in the tactical movement plan may be shaded or colored or otherwise marked with digital content in the FOV such that the primary BlueForce member can manage his activities with respect to the tactical plan.”).
15 Regarding claim 12, Border in view of Sugden teaches the method of claim 11, wherein setting the rendering parameters for the CGR object (see claims 1 and 11 rejections above) is in accordance with a determination that the CGR object and the light superposition characteristic value collectively satisfy one or more contrast criteria (Border; [0320] reciting “The type of content may also be particular to the venue, or other location selection criteria, such that the person is more apt to view and/or interact with the content.”).
16 Regarding claim 15, Border in view of Sugden teaches the method of claim 11, further comprising (see claims 1 and 11 rejections above):
rendering the CGR object according to the rendering parameters; and displaying the CGR object on the see-through display ([0176] reciting “Generally, in the present disclosure, instrument stroke parameter changes may be referred to as a change in line type, line weight, tip type, brush type, brush width, brush pressure, color, and other forms of writing, coloring, painting, and the like.”; [0275] reciting “The tree and the building are actually in the environment and the text is displayed in the see-through display such that it appears overlaid on the environment. In addition to text information such as, for example, instructions and weather information, some augmented reality information is shown that relates to nearby objects in the environment.”).
17 Regarding claim 16, Border in view of Sugden teaches the method of claim 1 (see claim 1 rejection above), wherein the display correction value is also a function of display characteristics of the see-through display (Border; [0103] reciting “In situations where the HWC 102 has integrated computer displays the displays may be configured as see-through displays such that the digital imagery can be overlaid with respect to the user's view of the environment 114. There are a number of see-through optical designs that may be used, including ones that have a reflective display (e.g. LCoS, DLP), emissive displays (e.g. OLED, LED, backlit LCD), hologram, TIR waveguides, and the like. In embodiments, lighting systems used in connection with the display optics may be solid state lighting systems, such as LED, OLED, quantum dot, quantum dot LED, etc. In addition, the optical configuration may be monocular or binocular. It may also include vision corrective optical components.”; [0119] reciting “The embodiment of FIG. 4 also includes a corrective wedge 420 to correct the effect of refraction of the image light 414 as it exits the TIR wedge 418. By including the corrective wedge 420 and providing a thin air gap 408 (e.g. 25 micron), the image light from the “on” pixels can be maintained generally in a direction along the optical axis of the field lens (i.e. the same direction as that defined by the image light 414) so it passes into the field lens and the lower optical module 204.”).
18 Regarding claim 17, Border in view of Sugden teaches the method of claim 16 (see claims 1 and 16 rejections above), wherein the display characteristics include a lens characteristic of a lens of the see-through display (Border; [0099] reciting “The glasses may be a fully developed computing platform, such as including computer displays presented in each of the lenses of the glasses to the eyes of the user. In embodiments, the lenses and displays may be configured to allow a person wearing the glasses to see the environment through the lenses while also seeing, simultaneously, digital imagery, which forms an overlaid image that is perceived by the person as a digitally augmented image of the environment, or augmented reality (“AR”).”; [0133] reciting “If the image light is unpolarized, a microlouvered film such as a privacy filter can be used to absorb the escaping image light while providing the user with a see-thru view of the environment.”).
19 Regarding claim 18, Border in view of Sugden teaches the method of claim 16 (see claims 1 and 16 rejections above), wherein the display characteristics indicates a distance between the see-through display and an eye of a user (Border; [0306] reciting “The presented object A 6018 is presented not only along the virtual target line but also at a focal plane B 6014 such that the content at position A 6012 in the FOV 6008 comes into focus by the first person when the first person's eye 6002 focuses at something in the surrounding environment at the focal plane B 6014 distance.”).
20 Claims 19-20 have similar limitations as of claim 1, therefore they are rejected under the same rationale as claim 1.
21 Claim(s) 6-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Border et al. (US 20160048021 A1) in view of Sugden et al. (US 20130141434 A1) as of claim 1, further in view of Yeung et al. (US 20210097757 A1).
22 Regarding claim 6, Border in view of Sugden teaches the method of claim 5, wherein changing the tint level of the see- through display (see claims 1 and 5 rejections above)
23 Border in view of Sugden does not explicitly teach wherein changing the tint level of the see- through display includes globally tinting the see-through display.
24 Yeung teaches wherein changing the tint level of the see- through display includes globally tinting the see-through display ([0018] reciting “Generally, head-mounted augmented reality display 100A may be a see-through display through which a user of the augmented reality display 100A can directly view the real-world environment in which the user is using the head-mounted augmented reality display 100A (as opposed to a digital image of the real-world environment).”; [0024] reciting “Generally, by generating the shader with a plain white texture and lens tint-colored vertices, the shader may be configured to be transparent or substantially transparent to a user of augmented reality display 100A.”; [0031] reciting “Generally, the addition of illumination effects to real-world objects in the environment in which the head mounted display 100A is operating may be rendered such that adjustments to lighting effects are made with respect to a global perceptual anchor object.”).
25 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Border in view of Sugden) to incorporate the teachings of Yeung to provide a method that can include global tinting based on the tinting methods provided by Yeung. Doing so would allow a type of shader generator to be configured with various colors of vertices to apply scenario-specific color tints to the real-world environment in which augmented reality display is being used as stated by Yeung ([0024] recited).
26 Regarding claim 7, Border in view of Sugden teaches the method of claim 5, wherein changing the tint level of the see-through display (see claims 1 and 5 rejections above)
27 Border in view of Sugden does not explicitly teach wherein changing the tint level of the see-through display includes tinting a portion of the see-through display that is less than an entirety of the see-through display.
28 Yeung teaches wherein changing the tint level of the see-through display includes tinting a portion of the see-through display that is less than an entirety of the see-through display ([Claim 5] reciting “…which the augmented reality display is operating such that illuminations against objects with higher reflectivity surfaces appears brighter than illuminations against objects with lower reflectivity surfaces, wherein illuminating the one or more real-world objects comprises decreasing values of one or more luminance channels for illuminated objects and increasing values of the one or more luminance channels for unilluminated objects such that illuminations against objects are perceived consistently through a tinted see-through optical display.”).
29 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Border in view of Sugden) to incorporate the teachings of Yeung to provide a method that includes tinting a see-through display that can be provided by Border in view of Sugden to be less than the taught display. Doing so would allow a type of shader generator to be configured with various colors of vertices to apply scenario-specific color tints to the real-world environment in which augmented reality display is being used as stated by Yeung ([0024] recited).
30 Regarding claim 8, Border in view of Sugden and Yeung teaches the method of claim 7, wherein the portion of the see-through display (see claims 1, 5, and 7 rejections above) corresponds to a location on the see-through display where the CGR object is to be displayed.
31 Yeung from claim 7 can further teach the limitations, specifically wherein the portion of the see-through display corresponds to a location on the see-through display where the CGR object is to be displayed ([Abstract] reciting “Embodiments provide for the rendering of illumination effects on real-world objects in augmented reality systems.”; [0016] reciting “vertices of the shader may be illuminated to render lighting effects on one or more polygons in the shader corresponding to locations of different real-world objects.”).
32 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Border in view of Sugden and Yeung) to incorporate additional teachings of Yeung to provide a location for the see-through display for the specific object to be displayed utilizing the displays and objects provided by Border in view of Sugden and Yeung. Doing so would allow a type of shader generator to be configured with various colors of vertices to apply scenario-specific color tints to the real-world environment in which augmented reality display is being used as stated by Yeung ([0024] recited).
33 Claim(s) 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Border et al. (US 20160048021 A1) in view of Sugden et al. (US 20130141434 A1) as of claim 1 and 11-12, further in view of Kim et al. (US 20140184577 A1).
34 Regarding claim 13, Border in view of Sugden teaches the method of claim 12 (see claims 1 and 11-12 rejections above), but does not explicitly teach wherein the one or more contrast criteria include a luminance contrast criterion.
35 Kim teaches wherein the one or more contrast criteria include a luminance contrast criterion ([Abstract] reciting “…an optical sensor configured to measure the amount of light, and a light shielding unit that is disposed on one surface of the transparent display unit and is configured to adjust transmittance on the basis of the amount of light.”; [0060] reciting “The visibility in the transparent display apparatus 100 may be measured using, for example, a contrast ratio of a video. In addition to the contrast ratio, a determination criterion of the visibility in the transparent display apparatus 100 may be different depending on various conditions such as luminance, color sensitivity, and resolution, or may be different depending on users. Hereinafter, a case where the determination criterion is the contrast ratio will be described below.”).
36 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Border in view of Sugden) to incorporate the teachings of Kim to provide a method for the criterion provided by Border in view of Sugden to include a type of luminance contrast. Doing so would allow the method to adjust transmittance on the basis of the amount of light as stated by Kim ([Abstract] recited).
37 Regarding claim 14, Border in view of Sugden teaches the method of claim 12 (see claims 1 and 11-12 rejections above), but does not explicitly teach wherein the one or more contrast criteria include a color contrast criterion.
38 Kim teaches wherein the one or more contrast criteria include a color contrast criterion ([0060] reciting “The visibility in the transparent display apparatus 100 may be measured using, for example, a contrast ratio of a video. In addition to the contrast ratio, a determination criterion of the visibility in the transparent display apparatus 100 may be different depending on various conditions such as luminance, color sensitivity, and resolution, or may be different depending on users. Hereinafter, a case where the determination criterion is the contrast ratio will be described below.”).
39 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Border in view of Sugden) to incorporate the teachings of Kim to provide a method for the criterion provided by Border in view of Sugden to include a type of color contrast. Doing so would allow the method to adjust transmittance on the basis of the amount of light as stated by Kim ([Abstract] recited).
Conclusion
40 The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Tusch et al. (US 20140168288 A1) teaches a display device screen containing brightness and strength, as well as including ambient light levels and averages/.
41 Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY TRAN LE whose telephone number is (571)272-5680. The examiner can normally be reached Mon-Thu: 7:30am-5pm; First Fridays Off; Second Fridays: 7:30am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHNNY T LE/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights