Prosecution Insights
Last updated: October 27, 2026
Application No. 19/411,321

VIRTUAL AND AUGMENTED REALITY SYSTEMS AND METHODS USING DISPLAY SYSTEM CONTROL INFORMATION EMBEDDED IN IMAGE DATA

Non-Final OA §101§103§DOUBLEPATENT
Filed
Dec 07, 2025
Priority
Feb 22, 2018 — continuation of 10/735,649 +4 more
Examiner
FARAGALLA, MICHAEL A
Art Unit
2624
Tech Center
2600 — Communications
Assignee
Magic Leap Inc.
OA Round
1 (Non-Final)
86%
Grant Probability
Favorable
1-2
OA Rounds
2y 0m
Est. Remaining
94%
With Interview

Examiner Intelligence

Grants 86% — above average
86%
Career Allowance Rate
872 granted / 1020 resolved
+23.5% vs TC avg
Moderate +8% lift
Without
With
+8.2%
Interview Lift
resolved cases with interview
Typical timeline
2y 11m
Avg Prosecution
32 currently pending
Career history
1058
Total Applications
across all art units

Statute-Specific Performance

§101
1.4%
-38.6% vs TC avg
§103
73.8%
+33.8% vs TC avg
§102
11.1%
-28.9% vs TC avg
§112
6.8%
-33.2% vs TC avg
Black line = Tech Center average estimate • Based on career data from 1020 resolved cases

Office Action

§101 §103 §DOUBLEPATENT
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Consider Claim 20, the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because: Claim 20 includes the language “more computer-readable storage media storing instructions which, when executed, cause at least one processor to perform operations for.” Paragraph 134 of the instant application states that “the controller 210 includes programming (e.g., instructions in a non-transitory computer-readable medium) that regulates the timing and provision of image information to the waveguides.” Neither the claim language, nor the specification specifically exclude non-transitory media. Double Patenting The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b). The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13. The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer. Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. Patent No. US 12,513,390. Although the claims at issue are not identical, they are not patentably distinct from each other because: Regarding Claim 1, claim limitations are fully anticipated by claim 1 of Patent No. US 12,513,390 as shown in the table below. Claims 16 and 20 are similarly rejected for being fully anticipated by either of independent claims 1, 15, 19, or 20 of Patent No. US 12,513,390. Claim 2 is anticipated by claim 2 of Patent No. US 12,513,390. Claim 3 is anticipated by claim 3 of Patent No. US 12,513,390. Claim 4 is anticipated by claim 4 of Patent No. US 12,513,390. Claim 5 is anticipated by claim 5 of Patent No. US 12,513,390. Claim 6 is anticipated by claim 6 of Patent No. US 12,513,390. Claim 7 is anticipated by claim 7 of Patent No. US 12,513,390. Claim 8 is anticipated by claim 8 of Patent No. US 12,513,390. Claim 9 is anticipated by claim 9 of Patent No. US 12,513,390. Claim 10 is anticipated by claim 10 of Patent No. US 12,513,390. Claim 11 is anticipated by claim 11 of Patent No. US 12,513,390. Claim 12 is anticipated by claim 12 of Patent No. US 12,513,390. Claim 13 is anticipated by claim 10 of Patent No. US 12,513,390. Claim 14 is anticipated by claim 14 of Patent No. US 12,513,390. Claim 15 is anticipated by claim 14 of Patent No. US 12,513,390. Claim 17 is anticipated by claim 13 of Patent No. US 12,513,390. Claim 18 is anticipated by claim 13 of Patent No. US 12,513,390. Claim 19 is anticipated by claim 14 of Patent No. US 12,513,390. 19/411,321 (Instant Application) Patent No. US 12,513,390 1. A display system comprising: a display configured to display image data, the image data comprising a current image data frame with a plurality of color components; a plurality of light sources configured to illuminate the display; and a display system controller configured to: receive the current image data frame, read control information embedded in the current image data frame, the control information comprising an indication to inactivate a selected color component within the current image data frame, and control at least a first control input for at least one of the plurality of light sources based on the embedded control information so as to inactivate the selected color component within the current image data frame, wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data. 1. A display system comprising: a display configured to display image data, the image data comprising a current image data frame with a plurality of color components; a plurality of light sources configured to illuminate the display; and a display system controller configured to: receive the current image data frame, read control information embedded in the current image data frame, the control information comprising an indication to inactivate a selected color component within the current image data frame, and control at least a first control input for at least one of the plurality of light sources based on the embedded control information so as to inactivate the selected color component within the current image data frame, wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data, and wherein the embedded control information further comprises advance frame display information which includes an indication to selectively re-activate the selected color component within a subsequent image data frame. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1, 4, 13-19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sung et al (Publication number: US 2013/0242049) in view of Holz (Publication number: US 2017/0345219). Consider Claim 1, Sung et al shows a display system (see figures 8 and 9), comprising: (a) A display configured to display image data (see figure 8 and 9; and paragraphs 28-30); (The advanced 3D method may play a 3D video with a sense of depth by synthesizing videos at the position of an arbitrary view utilizing video information and depth information which are transmitted during a playing process. When the advanced 3D method is used, a viewer may adjust the sense of depth, and a multiview display device may naturally synthesize a necessary number of views at successive viewpoints). (b) The image data comprising a current image data frame with a plurality of color components (see paragraph 30); (The MVD format may have first color information 100, second color information 200, and third color information 300, first depth information 105, second depth information 115, and third depth information 125). (c) A display system controller configured to: receive the current image data frame, read control information embedded in the current image data frame (see figure 8; and paragraphs 32-36, and 42); (The generated first and second MVC-compatible frames 460 and 490 may be encoded/decoded together through an MVC encoder and an MVC decoder). (d) Control information comprising an indication to inactivate a selected color component within the current image data frame (see paragraph 51); (A first MVC-compatible frame may be generated based on first color information and first depth information, and a second MVC-compatible frame may be generated based on second color information and second depth information. In the case of the LDV format, first and second MVC-compatible frames may be generated based on foreground textures and foreground depths). (e) A control at least a first control input for based on the embedded control information so as to inactivate the selected color component within the current image data frame (see figure 9 and 10; paragraphs 78-82); (The up-sampling unit 1040 may up-sample the frames separated by the frame separation unit 1020. The frames separated by the frame separation unit 1020 are frames which are down-sampled and synthesized by the first format conversion unit, and may have a resolution reduced to the half. Therefore, in order to increase the resolution of the frame, the up-sampling unit 1040 may perform up-sampling to reconstruct original advanced 3D contents). However, Sung et al does not specifically show a plurality of light sources configured to illuminate the display; wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data. In related art, Holz shows a plurality of light sources configured to illuminate the display; wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data (see paragraphs 116-120); (The world model can include representations of object portions (e.g. objects, edges of objects, prominent vortices) and potentially depth information when available from a depth sensor, depth camera or the like, within the viewpoint of the virtual or augmented reality head mounted sensor. The system can build the world model from image information captured by the cameras of the sensor). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the application to incorporate the depth information taught by Holz into the teaching of Sung et al in order to build a world model from image information (see Holz; paragraphs 116-120). Consider Claim 16, Sung et al shows a method for controlling a display system, the method performed by at least one processor of the display system (see figures 6 and 7), the method comprising: (a) Receiving a current image data frame to be presented through a display of the display system (see figure 8 and 9; and paragraphs 28-30); (The advanced 3D method may play a 3D video with a sense of depth by synthesizing videos at the position of an arbitrary view utilizing video information and depth information which are transmitted during a playing process. When the advanced 3D method is used, a viewer may adjust the sense of depth, and a multiview display device may naturally synthesize a necessary number of views at successive viewpoints). (b) The current image data frame including a plurality of color components (see paragraph 30); (The MVD format may have first color information 100, second color information 200, and third color information 300, first depth information 105, second depth information 115, and third depth information 125). (c) Reading control information embedded in the current image data frame (see figure 8; and paragraphs 32-36, and 42); (The generated first and second MVC-compatible frames 460 and 490 may be encoded/decoded together through an MVC encoder and an MVC decoder). (d) Control information comprising an indication to inactivate a selected color component within the current image data frame (see paragraph 51); (A first MVC-compatible frame may be generated based on first color information and first depth information, and a second MVC-compatible frame may be generated based on second color information and second depth information. In the case of the LDV format, first and second MVC-compatible frames may be generated based on foreground textures and foreground depths). (e) A control at least a first control input for based on the embedded control information so as to inactivate the selected color component within the current image data frame (see figure 9 and 10; paragraphs 78-82); (The up-sampling unit 1040 may up-sample the frames separated by the frame separation unit 1020. The frames separated by the frame separation unit 1020 are frames which are down-sampled and synthesized by the first format conversion unit, and may have a resolution reduced to the half. Therefore, in order to increase the resolution of the frame, the up-sampling unit 1040 may perform up-sampling to reconstruct original advanced 3D contents). However, Sung et al does not specifically show a plurality of light sources configured to illuminate the display; wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data. In related art, Holz shows a plurality of light sources configured to illuminate the display; wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data (see paragraphs 116-120); (The world model can include representations of object portions (e.g. objects, edges of objects, prominent vortices) and potentially depth information when available from a depth sensor, depth camera or the like, within the viewpoint of the virtual or augmented reality head mounted sensor. The system can build the world model from image information captured by the cameras of the sensor). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the application to incorporate the depth information taught by Holz into the teaching of Sung et al in order to build a world model from image information (see Holz; paragraphs 116-120). Consider Claim 20, Sung et al shows one or more computer-readable storage media storing instructions which, when executed, cause at least one processor to perform operations for controlling a display system (see figures 6 and 7), the operations comprising: (a) Receiving a current image data frame to be presented through a display of the display system (see figure 8 and 9; and paragraphs 28-30); (The advanced 3D method may play a 3D video with a sense of depth by synthesizing videos at the position of an arbitrary view utilizing video information and depth information which are transmitted during a playing process. When the advanced 3D method is used, a viewer may adjust the sense of depth, and a multiview display device may naturally synthesize a necessary number of views at successive viewpoints). (b) The current image data frame including a plurality of color components (see paragraph 30); (The MVD format may have first color information 100, second color information 200, and third color information 300, first depth information 105, second depth information 115, and third depth information 125). (c) Reading control information embedded in the current image data frame (see figure 8; and paragraphs 32-36, and 42); (The generated first and second MVC-compatible frames 460 and 490 may be encoded/decoded together through an MVC encoder and an MVC decoder). (d) Control information comprising an indication to inactivate a selected color component within the current image data frame (see paragraph 51); (A first MVC-compatible frame may be generated based on first color information and first depth information, and a second MVC-compatible frame may be generated based on second color information and second depth information. In the case of the LDV format, first and second MVC-compatible frames may be generated based on foreground textures and foreground depths). (e) A control at least a first control input for based on the embedded control information so as to inactivate the selected color component within the current image data frame (see figure 9 and 10; paragraphs 78-82); (The up-sampling unit 1040 may up-sample the frames separated by the frame separation unit 1020. The frames separated by the frame separation unit 1020 are frames which are down-sampled and synthesized by the first format conversion unit, and may have a resolution reduced to the half. Therefore, in order to increase the resolution of the frame, the up-sampling unit 1040 may perform up-sampling to reconstruct original advanced 3D contents). However, Sung et al does not specifically show a plurality of light sources configured to illuminate the display; wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data. In related art, Holz shows a plurality of light sources configured to illuminate the display; wherein the embedded control information is inserted into the current image data frame in place of a portion of the image data (see paragraphs 116-120); (The world model can include representations of object portions (e.g. objects, edges of objects, prominent vortices) and potentially depth information when available from a depth sensor, depth camera or the like, within the viewpoint of the virtual or augmented reality head mounted sensor. The system can build the world model from image information captured by the cameras of the sensor). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the application to incorporate the depth information taught by Holz into the teaching of Sung et al in order to build a world model from image information (see Holz; paragraphs 116-120). Consider claim 4, Holz shows a display system controller is configured to control a second control input for at least one of the plurality of light sources based on the control information embedded in the current image data frame (see paragraphs 116-120); (The world model can include representations of object portions (e.g. objects, edges of objects, prominent vortices) and potentially depth information when available from a depth sensor, depth camera or the like, within the viewpoint of the virtual or augmented reality head mounted sensor. The system can build the world model from image information captured by the cameras of the sensor). Consider Claims 13, and 17, Sung et al shows that the control information is embedded in the current image data frame in at least one substituted row of the image data (see figure 8 and 9; and paragraphs 28-30); (The advanced 3D method may play a 3D video with a sense of depth by synthesizing videos at the position of an arbitrary view utilizing video information and depth information which are transmitted during a playing process. When the advanced 3D method is used, a viewer may adjust the sense of depth, and a multiview display device may naturally synthesize a necessary number of views at successive viewpoints). Consider Claims 14, and 18, Sung et al shows that the control information is embedded in the current image data frame as one or more substituted values in one or more pixels of the image data (see figure 8 and 9; and paragraphs 28-30); (The advanced 3D method may play a 3D video with a sense of depth by synthesizing videos at the position of an arbitrary view utilizing video information and depth information which are transmitted during a playing process. When the advanced 3D method is used, a viewer may adjust the sense of depth, and a multiview display device may naturally synthesize a necessary number of views at successive viewpoints). Consider Claims 15, and 19, Sung et al shows that the display system controller is further configured to remove the embedded control information from the current image data frame before the current image data frame is displayed (see paragraph 51); (A first MVC-compatible frame may be generated based on first color information and first depth information, and a second MVC-compatible frame may be generated based on second color information and second depth information. In the case of the LDV format, first and second MVC-compatible frames may be generated based on foreground textures and foreground depths). Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Sung et al (Publication number: US 2013/0242049) in view of Holz (Publication number: US 2017/0345219) in view of Send et al (Publication number: US 2015/0286340). Consider Claim 2, Sung et al in view of Holz do not specifically show a plurality of spatial light modulators configured to modulate light from the plurality of light sources with the image data, wherein the display system controller is further configured to control at least a second control input for at least one of the plurality of spatial light modulators based on the embedded control information, wherein the second control input comprises a reduced-power setting for a spatial light modulator corresponding to the selected color component. In related art, Send et al shows a plurality of spatial light modulators configured to modulate light from the plurality of light sources with the image data, wherein the display system controller is further configured to control at least a second control input for at least one of the plurality of spatial light modulators based on the embedded control information, wherein the second control input comprises a reduced-power setting for a spatial light modulator corresponding to the selected color component (see paragraphs 31, 53, and 457); (the at least one optical sensor has at least one sensor region, wherein the sensor signal of the optical sensor is dependent on an illumination of the sensor region by the light beam, wherein the sensor signal, given the same total power of the illumination, is dependent on a width of the light beam in the sensor region, wherein the evaluation device preferably is adapted to determine the width by evaluating the sensor signal). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the application to modify Sung and Holz with Send where the second controller must be a feature of the modified invention to function with SLMs, since such a modification increases reliability of the device (see Send; paragraph 15). Allowable Subject Matter Claims 3, and 5-12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims; and if Applicant overcomes the Double Patenting rejection of record. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL A FARAGALLA whose telephone number is (571)270-1107. The examiner can normally be reached Mon-Fri 8:00-5:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Eason can be reached at 571-270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MICHAEL A FARAGALLA/Primary Examiner, Art Unit 2624 09/27/2026
Read full office action

Prosecution Timeline

Dec 07, 2025
Application Filed
Sep 30, 2026
Non-Final Rejection mailed — §101, §103, §DOUBLEPATENT (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12763001
DISPLAY PANEL DRIVING CIRCUIT AND DISPLAY DEVICE INCLUDING THE SAME
1y 10m to grant Granted Oct 13, 2026
Patent 12751188
LIGHT EMITTING DISPLAY DEVICE
1y 7m to grant Granted Sep 29, 2026
Patent 12724509
INPUT SENSING UNIT AND DISPLAY DEVICE INCLUDING THE SAME
3y 0m to grant Granted Sep 01, 2026
Patent 12726257
ELECTRONIC DEVICE COMPRISING A METASURFACE
2y 12m to grant Granted Sep 01, 2026
Patent 12717423
INPUT METHOD EDITOR FOR REMOTE APPLICATIONS
4y 11m to grant Granted Aug 25, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
86%
Grant Probability
94%
With Interview (+8.2%)
2y 11m (~2y 0m remaining)
Median Time to Grant
Low
PTA Risk
Based on 1020 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month