DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114.
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/05/2026 has been entered. Claims 1-19, filed 06/05/2026 are presented for examination.
Response to Arguments
Applicant’s arguments with respect amended claims 1, 6, 19 filed on 06/05/2026 have been considered but they are not persuasive.
However, examiner found some amended limitations are taught by references previous introduced.
In Remark page 8, lines 4-8, applicant argued As indicated in the Final Office Action (see pages 14-18), Edgren does not teach "the control information is embedded in the rendered imagery by at least one of: i) replacing a row or a column in a frame of the rendered imagery with the embedded control information, ii) replacing one or more values in one or more pixels of the rendered imagery with the embedded control information, and iii) adding an additional row or column in a frame of the rendered
imagery, the additional row or column including the embedded control information."
The examiner respectfully disagrees with Applicant’s argument. In fact, Edgren in view of Spring et al. (U.S. 2020/0312026 A1) can be used to teach the amendment. In paragraph [0071], Spring discloses “as the 3D scene is rendered, any contextual information that has been added to the scene can be shown as icons, links, etc. embedded in the 3D scene such as shown in FIGS. 4E through 4K” and [0097] “with reference now to FIG. 4H, a screen shot 450 of an optimized 3D scene rendered with contextual information identifiers 441-443 and their associated frames 451-453 are presented on the display 116” Spring teaches the control information is embedded by replacing (adding) a row or a column in a frame (rows, columns in frames 451-453, Fig. 4H) of the rendered image with the embedded control information.
Independent claims 6 and 19 recite similar elements as independent claim 1 and are rejected as the same above explanation.
The rejection of claim 19 under 35 U.S.C. §101 is withdrawn in of Applicant’s amended in claim 19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filling date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-11, 13-15 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable by Edgren et al. (U.S. 2014/0160100 A1) in view of Spring et al. (U.S. 2020/0312026 A1).
Regarding Claim 1 (Currently amended), Edgren discloses a virtual or augmented reality display system (Edgren, [0007] “a user interface system for adapting a graphic visualization of a virtual element on a display” Edgren teaches a virtual display system comprising:
a sensor configured to determine one or more characteristics of ambient lighting (Edgren, [0012] “the user interface system determines surrounding light characteristic of surrounding light associated with the display, the surrounding light from one or several sensors” and [0015] “the surrounding light may comprise ambient light” Edgen teaches a sensor determines one or more characteristics of ambient lighting (a surrounding light);
a display configured to present rendered imagery (Edgren, ([0040] “adapting a graphic visualization of a virtual element on a display to render a corresponding fictitious physical element, there will be disclosed that rendering may change dynamically with changing surrounding light conditions” Edgren teaches a display to a corresponding fictitious physical element on image;
a display controller (Edgren, ([0041] “a display controller node” Edgren teaches a controller) configured to:
receive the rendered imagery that includes virtual content to be presented through the display, wherein the rendered imagery further includes control information embedded in the rendered imagery (Edgren, [0018] “The shadow property may for instance relate to relevant refinements of the graphic visualization to embed a virtual shadow resembling a shadow of a corresponding fictitious physical element subjected to the corresponding surrounding light conditions” and [0019] “by taking the intensive directed light direction property, and subsequently direction of determined intensive directed light, into consideration when determining the shadow property in the rendering process as suggested by said embodiment, the graphic visualization may be adjusted such that the embedded virtual shadow resemble the shadow of the fictitious physical element should the fictitious physical element have been subjected to the intensive directed light” Edgren teaches receive the rendered imagery includes a virtual content (e.g., a virtual shadow) to be presented through the display, further includes control information embedded in the rendered imagery (e.g., the graphic visualization is adjusted by the embedded virtual shadow to resemble the shadow of the fictitious physical element under the intensive directed light), [[and]] wherein the control information is embedded in a portion of the rendered imagery that is separate from the virtual content to be presented through the display (Edgren, Fig. 3, [0056] “the adaptation of the graphic visualization 13 may be based on at least the shadow property, such that a virtual shadow of the virtual element 14 is applied in the graphic visualization 13 to resemble a shadow of the fictitious physical element should the fictitious physical element have been subjected to the intensive directed light 17…when the shadow property additionally is based on at least the obstacle distance property and/or the obstacle shape property, be based on the shadow property such that a virtual shadow 42 of the virtual element 14 is applied in the graphic visualization 13 to resemble a shadow 22 of the fictitious physical element should the fictitious physical element have been subjected to the intensive directed light 17” Edgren teaches the control information is embedded in a portion of the rendered image e.g., the shadow property of a virtual shadow 42, (Fig. 3), a finger shadow of the virtual element 14 (a portion of dash board in rendered imagery, Fig. 5), is separated from the virtual content (the virtual shadow of the virtual element 14, Fig. 5e) to be presented through the display under the directed light 17 (Fig. 3), and
read the control information embedded in the rendered imagery (Edgren, [0043] “in the determined direction 18 of the intensive directed light. FIG. 3. The obstacle 15 is here represented by a hand, and particularly an index finger, , approaching the display 3 in a determined direction 20. The obstacle 15 is positioned at a determined distance 19 from the display 3. A shadow 22 of the obstacle 15 is partly casted on the display 3, and subsequently a part thereof on the graphic visualization 13” Edgren teaches the Fig. 3 shows read (apply) the control information embedded in the rendered imagery e.g., determining direction 18 of the directed light, the obstacle hand 15 with particular an index finger approaching the display 3 at a distance 19. A shadow finger 22 of the obstacle hand 15 is partly casted/embedded on the display 3 in the rendered imagery;
based on the control information, adjust one or more characteristics of a virtual content to match the one or more characteristics of the ambient lighting (Edgen, [0007] “The user interface system determines surrounding light characteristics of surrounding light associated with the display, and determines light adjusted characteristics of the virtual element based on the surrounding light characteristics” and [0015] “the surrounding light may comprise ambient light” and [0042] “The user interface system 2 may comprise, the processor 10” Edgen teaches based on the control information, adjust characteristics of a virtual content (the shadow of the finger) to match the characteristic of the ambient light (the surrounding/background light) wherein the display controller is configured to adjust the one or more characteristics of the virtual content after the virtual content has been rendered (Edgen, [0055] “the adaptation of the graphic visualization 13 may comprise selecting a graphic visualization profile to represent the graphic visualization 13 from a set of predefined graphic visualization profiles. The best-fitting profile, i.e. pre-rendered graphic visualization, constituting the best match is selected from the predefined set, which set for instance is stored in memory 11” and [0058] “in Action 403 determines light adjust characteristics of the virtual element 14 based on the surrounding light characteristics. Then, when the user interface 2 adapts the graphic visualization 13 based on the light adjusted characteristics” Edgen teaches adjust characteristics of the virtual object from a selecting predefined graphic visualization profile, i.e., pre-rendered graphic visualization (the virtual object has been rendered), and
instruct the display to present the virtual content with the adjusted one or more characteristics (Edgren, [0019] “by taking the intensive directed light direction property, and subsequently direction of determined intensive directed light, into consideration when determining the shadow property in the rendering process as suggested by said embodiment, the graphic visualization may be adjusted such that the embedded virtual shadow resemble the shadow of the fictitious physical element should the fictitious physical element have been subjected to the intensive directed light” Edgren teaches instruct the display includes a virtual content (e.g., a virtual shadow) to be presented through the display, further includes control information embedded in the rendered imagery (e.g., the graphic visualization is adjusted by the virtual content with characteristics (the embedded virtual shadow) to resemble the shadow of the fictitious physical element under the intensive directed light).
However, Edgren does not explicitly teach wherein the control information is embedded in the rendered imagery by at least one of:
i) replacing a row or a column in a frame of the rendered imagery with the
embedded control information, ii) replacing one or more values in one or more pixels of the rendered imagery with the embedded control information, and iii) adding an additional row or column in a frame of the rendered imagery, the additional row or column including the embedded control information;
Spring teaches the control information is embedded in the rendered imagery by i) replacing a row or a column in a frame of the rendered imagery with the embedded control information (Spring, [0071] “as the 3D scene is rendered, any contextual information that has been added to the scene can be shown as icons, links, etc. embedded in the 3D scene such as shown in FIGS. 4E through 4K” and [0097] “with reference now to FIG. 4H, a screen shot 450 of an optimized 3D scene rendered with contextual information identifiers 441-443 and their associated frames 451-453 are presented on the display 116” Spring teaches the control information is embedded by replacing (adding) a row or a column in a frame (rows, columns in frames 451-453, Fig. 4H) of the rendered image with the embedded control information.
Edgren and Spring are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Edgren to combine with embedded rows/columns in a frame (as taught by Spring) in order to apply the embedded rows/columns in a frame because Spring can provide the control information is embedded by replacing (adding) a row or a column in a frame (rows, columns in frames 451-453, Fig. 4H) of the rendered image with the embedded control information (Spring, Fig. 4H, [0071], [0097]). Doing so, it may provide user to see where additional contextual information may be useful, and enables the user to point the camera to the desired region to obtain the contextual information (Spring, [0044]).
Regarding Claim 2, a combination of Edgren and Spring discloses the virtual or augmented reality display system of Claim 1, wherein the one or more characteristics of the ambient lighting comprise a brightness of the ambient lighting (Edgren, [0015] “the surrounding light may comprise ambient light” and [0050] “light adjusted characteristics of the virtual element 14 based on the surrounding light characteristics. The light adjusted characteristics may comprise an illumination property” Edgren teaches the ambient lighting (the surrounding light) includes a brightness (illumination property) of the ambient lighting.
Regarding Claim 3, a combination of Edgren and Spring discloses the virtual or augmented reality display system of Claim 1, wherein the one or more characteristics of the ambient lighting comprise a hue of the ambient lighting (Edgren, [0015] “the surrounding light may comprise ambient light” and [0050] “light adjusted characteristics of the virtual element 14 based on the surrounding light characteristics. The light adjusted characteristics may comprise a hue property” Edgren teaches the ambient lighting (the surrounding light) includes hue (hue property) of the ambient lighting.
Regarding Claim 4, a combination of Edgren and Spring discloses the virtual or augmented reality display system of Claim 1, wherein the one or more characteristics of the virtual object content comprise a brightness of the virtual content (Edgren, [0059] “Properties such as shadows, illuminations, hues,…of the virtual element 14, which may have effect on different areas of the graphic visualization 13, may have been adapted such that the graphic visualization 13 realistically renders a corresponding fictitious physical element” Edgren teaches a characteristic of the virtual object (virtual element 14) includes a brightness (an illumination) of the virtual content.
Regarding Claim 5, a combination of Edgren and Spring discloses the virtual or augmented reality display system of Claim 1, wherein the one or more characteristics of the virtual object content comprise a hue of the virtual content (Edgren, [0059] “Properties such as shadows, illuminations, hues,…of the virtual element 14, which may have effect on different areas of the graphic visualization 13, may have been adapted such that the graphic visualization 13 realistically renders a corresponding fictitious physical element” Edgren teaches a characteristic of the virtual object (virtual element 14) includes a hue of the virtual content.
Regarding Claim 6 (Currently amended), a combination of Edgren and Spring discloses a method (Edgren, [0007] “a method performed by a user interface system for adapting a graphic visualization of a virtual element on a display” Edgren teaches a method of a virtual display system, in a virtual or augmented reality display system, the method comprising:
receiving the rendered imagery that includes virtual content to be presented through the display, wherein the rendered imagery further includes control information embedded in the rendered imagery, [[and]] wherein the control information is embedded in a portion of the rendered imagery that is separate from the virtual content to be presented through the display, and wherein the control information is embedded in the rendered imagery by at least one of: i) replacing a row or a column in a frame of the rendered imagery with the
embedded control information, ii) replacing one or more values in one or more pixels of the rendered imagery with the embedded control information, and iii) adding an additional row or column in a frame of the rendered imagery, the additional row or column including the embedded control information;
reading the control information embedded in the rendered imagery;
receiving one or more characteristics of ambient lighting from a sensor of the display system;
based on the control information, adjusting one or more characteristics of the virtual content to match the one or more characteristics of the ambient lighting, wherein the one or more characteristics of the virtual content are adjusted after the virtual content has been rendered; and
instructing the display to present the virtual content with the adjusted one or more characteristics.
Claim 6 is substantially similar to claim 1 is rejected based on similar analyses.
Regarding Claim 7, a combination of Edgren and Spring discloses the method of Claim 6, wherein the one or more characteristics of the ambient lighting comprise a brightness of the ambient lighting.
Claim 7 is substantially similar to claim 2 is rejected based on similar analyses.
Regarding Claim 8, a combination of Edgren and Spring discloses the method of Claim 6, wherein the one or more characteristics of the ambient lighting comprise a hue of the ambient lighting.
Claim 8 is substantially similar to claim 3 is rejected based on similar analyses.
Regarding Claim 9, a combination of Edgren and Spring discloses the method of Claim 6, wherein the one or more characteristics of the virtual content comprise a brightness of the virtual content.
Claim 9 is substantially similar to claim 4 is rejected based on similar analyses.
Regarding Claim 10, a combination of Edgren and Spring discloses the method of Claim 6, wherein the one or more characteristics of the virtual content comprise a hue of the virtual content.
Claim 10 is substantially similar to claim 5 is rejected based on similar analyses.
Regarding Claim 11, the method of Claim 6, Edgren does not explicitly teach wherein the control information is embedded in the rendered imagery by replacing a row or a column in a frame of the rendered imagery with the embedded control information.
However, Spring teaches the control information is embedded in the rendered imagery by replacing a row or a column in a frame of the rendered imagery with the embedded control information (Spring, [0071] “as the 3D scene is rendered, any contextual information that has been added to the scene can be shown as icons, links, etc. embedded in the 3D scene such as shown in FIGS. 4E through 4K” and [0097] “with reference now to FIG. 4H, a screen shot 450 of an optimized 3D scene rendered with contextual information identifiers 441-443 and their associated frames 451-453 are presented on the display 116” Spring teaches the control information is embedded by replacing (adding) a row or a column in a frame (rows, columns in frames 451-453, Fig. 4H) of the rendered image with the embedded control information.
Edgren and Spring are combinable see rationale in claim 1.
Regarding Claim 13, the method of Claim 6, Edgren does not explicitly teach wherein the control information is embedded in the rendered imagery by adding an additional row or column in a frame of the rendered imagery, the additional row or column including the embedded control information.
However, Spring teaches the control information is embedded in the rendered imagery by adding an additional row or column in a frame of the rendered imagery, the additional row or column including the embedded control information (Spring, [0071] “as the 3D scene is rendered, any contextual information that has been added to the scene can be shown as icons, links, etc. embedded in the 3D scene such as shown in FIGS. 4E through 4K” and [0097] “with reference now to FIG. 4H, a screen shot 450 of an optimized 3D scene rendered with contextual information identifiers 441-443 and their associated frames 451-453 are presented on the display 116” Spring teaches the control information is embedded by adding a row or a column in a frame (rows, columns in frames 451-453, Fig. 4H) of the rendered image with the embedded control information.
Edgren and Spring are combinable see rationale in claim 1.
Regarding Claim 14, the method of Claim 6, Edgren does not explicitly teach further comprising removing the control information from the rendered imagery.
However, Spring teaches removing the control information from the rendered imagery (Spring, [0059] “FIG. 4B. Screen shot 405 includes crosshairs 402, contextual information capture 407, scan start/stop 401, and large ring 408” and [0060] “small ring 412 is used to indicate to the user that the motion has been reduced to within the bounds of contextual information capturing device's ability to capture a higher resolution image). Once large ring 408 disappears and small ring 412 appears the user can select contextual information” Spring teaches removing the control information (e.g., the large ring 408, the contextual information) in order to use the small ring 412 to capture a higher resolution image (Figs, 4B, 4C).
Edgren and Spring are combinable see rationale in claim 1.
Regarding Claim 15, a combination of Edgren and Spring discloses the virtual or augmented reality display system of Claim 1, wherein the control information is embedded in the rendered imagery by replacing a row or a column in a frame of the rendered imagery with the embedded control information.
Claim 15 is substantially similar to claim 11 is rejected based on similar analyses.
Regarding Claim 17, a combination of Edgren and Spring discloses the virtual or augmented reality display system of Claim 1, wherein the control information is embedded in the rendered imagery by adding an additional row or column in a frame of the rendered imagery, the additional row or column including the embedded control information.
Claim 17 is substantially similar to claim 13 is rejected based on similar analyses.
Regarding Claim 18, a combination of Edgren and Spring discloses the virtual or augmented reality display system of Claim 1, wherein the display controller is further configured to remove the control information from the rendered imagery.
Claim 18 is substantially similar to claim 14 is rejected based on similar analyses.
Regarding Claim 19 (Currently amended), a combination of Edgren and Spring discloses one or more non-transitory computer-readable storage media (Edgren, [0042] “software and/or firmware, e.g. stored in a memory such as the memory 11, that when executed by the one or more processors such as the processor 10” storing instructions which, when executed, cause at least one processor to perform operations for presenting rendered imagery through a display of a virtual or augmented reality display system, the operations comprising:
receiving the rendered imagery that includes virtual content to be presented through the display, wherein the rendered imagery further includes control information embedded in the rendered imagery, [[and]] wherein the control information is embedded in a portion of the rendered imagery that is separate from the virtual content to be presented through the display, and wherein the control information is embedded in the rendered imagery by at least one of: i) replacing a row or a column in a frame of the rendered imagery with the
embedded control information, ii) replacing one or more values in one or more pixels of the rendered imagery with the embedded control information, and iii) adding an additional row or column in a frame of the rendered imagery, the additional row or column including the embedded control information;
reading the control information embedded in the rendered imagery; receiving one or more characteristics of ambient lighting from a sensor of the display system;
based on the control information, adjusting one or more characteristics of the virtual content to match the one or more characteristics of the ambient lighting, wherein the one or more characteristics of the virtual content are adjusted after the virtual content has been rendered; and
instructing the display to present the virtual content with the adjusted one or more characteristics.
Claim 19 is substantially similar to claim 1 is rejected based on similar analyses.
Claims 12, 16 are rejected under 35 U.S.C. 103 as being unpatentable by Edgren (U.S. 2014/0160100 A1) in view of Spring et al. (U.S. 2020/0312026 A1) and further in view of Spitzer et al. (U.S. 2018/0136720 A1).
Regarding Claim 12, the method of Claim 6, a combination Edgren and Spring does not explicitly teach wherein the control information is embedded in the rendered imagery by replacing one or more values in one or more pixels of the rendered imagery with the embedded control information.
However, Spitzer teaches the control information is embedded in the rendered imagery by replacing one or more values in one or more pixels of the rendered imagery with the embedded control information (Spitzer, [0092] “the rendering device 1402 may transmit the query and receive the indicator in response via a control channel implemented as parameters embedded in a header or other field of a pixel stream” and [0079] “FIG. 12, an image region 1202 having a resolution of 10×16 pixels is to be combined with an image region 1204 having a resolution of 6×10 pixels, diagram 1206, the pixels of the first row 1208 (row 0) of the base image region 1202 are extracted, as is a set 1220 of the first 6 pixels from row 0 of the non-base image region 1204, and these pixels are combined to generate the illustrated row 1212 (row 0) of twenty-two pixels for a resulting combined pixel array” Spitzer teaches the control information (a control channel) is embedded in the rendered imagery by replacing (adding) values in pixels of the render imagery.
Edgren, Spring and Spitzer are combinable because they are from the same field of endeavor, system and method for image processing and try to solve similar problems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made for modifying the method of Edgren to combine with replacing one or more values in one or more pixels of the rendered imagery (as taught by Spitzer) in order to replacing (adding) one or more values in one or more pixels of the rendered imagery with the embedded control information because Spitzer can provide the control information (a control channel) is embedded in the rendered imagery by replacing (adding) values in pixels of the render imagery (Spitzer, Fig. 12, [0079], [0092]). Doing so, it may provide efficiently allocating pixel computation processes between the rendering device and the display controller (Spitzer, [0025]).
Regarding Claim 16, a combination of Edgren, Spring and Spitzer discloses the virtual or augmented reality display system of Claim 1, wherein the control information is embedded in the rendered imagery by replacing one or more values in one or more pixels of the rendered imagery with the embedded control information.
Claim 16 is substantially similar to claim 12 is rejected based on similar analyses.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KHOA VU whose telephone number is (571)272-5994. The examiner can normally be reached 8:00- 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KHOA VU/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights