DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/15/2026 has been entered.
Application Status
This office action is responsive to the amendments (Dated: 05/11/2026), pertaining to Application No.:18/584,239, filed on 02/22/2024 (Priority Date: 03/02/2023).
Claims 1-19 are pending and presented for examination.
This action has been made NON-FINAL.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Arguments
Applicant's arguments filed 05/11/2026 have been fully considered but they are not persuasive.
The Applicant alleged the following: “With the above configuration, the overall appearance of the displayed images is more natural to the viewer, particularly the VR-generated background portions that are "influenced" by the real-life light sources appearing in the image. These features of the present invention are not disclosed in the cited prior art.” The examiner is not persuaded. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., displayed images is more natural to the viewer, particularly the VR-generated background portions that are "influenced" by the real-life light sources appearing in the image) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Accordingly, the examiner maintains the rejection.
The Applicant alleged the following: “In particular, Kim, Tomite, and Tsuge, either alone or in combination, do not teach or suggest, the feature of "acquiring information relating to an irradiation range of a light source existing in real space captured by an image capture apparatus and generating, based on a determination of a region of a virtual display surface intersecting the irradiation range of the light source, the image representing the virtual space that reflects an influence exerted by light from the light source existing in the real space as if that light source existed in the virtual space," as recited in Applicant's amended claim 1.” The examiner is not persuaded. The combination of Kim, Tomite and Tsuge discloses the Applicant’s claim language of “acquiring information relating to an irradiation range of a light source existing in real space captured by an image capture apparatus.” More specifically, Kim’s teachings in Figure 1; Paragraphs 0022-0023; 0032; 0074-0077 discloses the Applicant’s claim limitations of “an acquisition unit configured to acquire information” and “of a light source existing in real space captured by an image capture apparatus.” Tsuge discloses an irradiation range in Paragraphs 0149; 0172; 0213; 0584. Moreover, Kim, Tomite and Tsuge discloses the Applicant’s claim language of “generating, based on a determination of a region of a virtual display surface intersecting the irradiation range of the light source, the image representing the virtual space that reflects an influence exerted by light from the light source existing in the real space as if that light source existed in the virtual space”. More specifically, Kim discloses the generation unit generates in Figure 2; Paragraphs 0022-0023. Tsuge goes on to disclose “based on a determination of a region of a virtual display interface intersecting with irradiation range of the light source” in Paragraphs 0149; 0172; 0213; 0584. Tomite goes on to discloses “the image representing the virtual space that reflects an influence” in Paragraphs 0017; 0052 and “existing in the real space as if that light source existed in the virtual space” in Paragraphs 0017; 0052. MPEP § 2106 states Office personnel are to give claims their broadest reasonable interpretation in light of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 1023, 1027-28 (Fed Cir. 1997). Accordingly, the examiner maintains the rejection.
The Applicant alleged the following: ”Therefore, Kim does not disclose or suggest the features of "acquiring information relating to an irradiation range of a light source existing in real space captured by an image capture apparatus and generating, based on a determination of a region of a virtual display surface intersecting the irradiation range of the light source, the image representing the virtual space that reflects an influence exerted by light from the light source existing in the real space as if that light source existed in the virtual space," as recited in Applicant's amended claim 1.” The examiner is not persuaded. The Applicant’s representative is rehashing arguments already presented above. Accordingly, the examiner maintains the rejection.
The Applicant alleged the following: “Therefore, Tomite does not disclose, teach, or suggest the features of "acquiring information relating to an irradiation range of a light source existing in real space captured by an image capture apparatus and generating, based on a determination of a region of a virtual display surface intersecting the irradiation range of the light source, the image representing the virtual space that reflects an influence exerted by light from the light source existing in the real space as if that light source existed in the virtual space," as recited in Applicant's amended claim 1.” The examiner is not persuaded. The Applicant’s representative is rehashing arguments already presented above. Accordingly, the examiner maintains the rejection.
The Applicant alleged the following: “Thus, Tsuge is directed to an EL display apparatus and describes a current-driving circuit for driving EL light-emitting elements. Tsuge merely describes a timing operation in which specific transistors are turned on or off such that a driving transistor operates in a saturation region, thereby maintaining a constant current value. That is, Tsuge's disclosure is limited to electrical driving control of display hardware, including transistor operation and current stability. Tsuge does not disclose or suggest "a light source existing in real space" or acquiring information relating to a real-space light source". Further, Tsuge does not disclose or suggest acquiring an irradiation range of a real-space light source, determining a region of a virtual display surface intersecting such an irradiation range, or generating an image representing a virtual space that reflects an influence exerted by the real-space light source as if that light source existed in the virtual space, as recited in amended claim 1. In fact, Tsuge's teachings are unrelated to image generation or illumination influence in a virtual space context.” The examiner is not persuaded. The Applicant’s representative is rehashing arguments already presented above. Accordingly, the examiner maintains the rejection.
The Applicant alleged the following: “Accordingly, Kim, Tomite, and Tsuge, either alone or in combination, do not disclose, teach, or suggest the features of "acquiring information relating to an irradiation range of a light source existing in real space captured by an image capture apparatus and generating, based on a determination of a region of a virtual display surface intersecting the irradiation range of the light source, the image representing the virtual space that reflects an influence exerted by light from the light source existing in the real space as if that light source existed in the virtual space," as recited in amended claim 1.” The examiner is not persuaded. The Applicant’s representative is rehashing arguments already presented above. Accordingly, the examiner maintains the rejection.
The Applicant alleged the following: “Thus, even if one were to combine Kim's general content-authoring lighting estimation (as characterized in the Office Action), Tomite's MR virtual light placement, and Tsuge's EL driving circuitry, the combination would still not result in the features of "calculating, for each pixel of the background image, an amount of variation in saturation and luminance as the influence exerted by the real light source, based on a luminance distribution in the region of the virtual display surface intersecting the irradiation range of the real light source and based on a color temperature of the real light source". Accordingly, claims 18 and 19 are also patentably distinct over the cited references.” The examiner is not persuaded. The combination of Kim, Tomite, and Tsuge discloses “wherein the generation unit calculates, for each pixel of the background image” in Tomite Paragraphs 0055-0058 of Tomite. Tsuge discloses “an amount of variation in saturation” in Paragraph 0149. Kim discloses “luminance as the influence exerted by the real light source” in Paragraphs 0022-0023. Tsuge discloses “wherein said influence is based on a luminance distribution” and “in the region of the virtual display surface intersecting the irradiation range” in Paragraphs 0169; 0172; 0584. Kim discloses “the real light source” in Paragraphs 0022-0023 and Tsuge discloses “based on a color temperature “in Paragraph 0213. Accordingly, the examiner maintains the rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim, US 20100033484 in view of Tomite, US 20080024523 and in further view of Tsuge, US 20050030264.
Claim 1:
Kim discloses an image processing apparatus (See Kim Abstract; Paragraph 0012). However, Kim fails to disclose an influence. This feature is disclosed in paragraph 0052 of Tomite. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have further modified Kim by the teachings of Tomite to enable improved generating images by incorporating influences, more effectively (See Tomite Field of Invention).
Additionally, the combination of Kim and Tomite failed to disclose “a determination of a region of a virtual display interface intersecting with irradiation range of the light source.” However, this feature is disclosed in paragraphs 0149; 0172; 0213; 0584 of Tsuge. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have further modified Kim and Tomite by the teachings of Tsuge’s image display system to enable improved light emitting display panel, more effectively (See Tsuge’s Technical Field).
As modified:
The combination of Kim, Tomite and Tsuge discloses the following:
one or more processors that execute a program stored in a memory (See Kim Figure 1; Paragraphs 0022-0023) and thereby function as:
an acquisition unit configured to acquire information (See Kim Figure 1; Paragraphs 0022-0023; 0074-00771) relating to an irradiation range (See Tsuge Paragraphs 0149; 0172; 0213; 0584) of a light source existing in real space (See Kim Paragraphs 0022-00232) captured by an image capture apparatus (See Kim Paragraphs 0023-0024; 00323);
and a generation unit configured to generate (See Kim Figure 2; Paragraphs 0022-00234), based on a three-dimensional model of a virtual space (See Kim Paragraphs 0022-00235), an image of the virtual space for displaying on a display apparatus disposed in the real space (See Kim Paragraph 00226),
wherein the generation unit generates (See Kim Figure 2; Paragraphs 0022-00237), based on a determination of a region of a virtual display interface intersecting with irradiation range of the light source (See Tsuge Paragraphs 0149; 0172; 0213; 0584), the image representing the virtual space that reflects an influence (See Tomite Paragraphs 0017; 00528) exerted by light from the light source (See Kim Paragraphs 0022-00239) existing in the real space as if that light source existed in the virtual space (See Tomite Paragraphs 0017; 0052).
Claim 2:
The combination of Kim, Tomite, and Tsuge discloses a determination unit (See Kim Figure 1; Paragraphs 0022-0023) configured to determine whether the light existing in the real space (See Tomite Paragraphs 0016-0018; 0033-0034) and emitted by the light source (See Kim Paragraphs 0022-0023) exerts an influence on the virtual space (See Tomite Paragraph 0052), wherein, if it is determined that the light exerts an influence on the virtual space (See Tomite Paragraph 0052), the generation unit generates the image in which the influence (See Tomite Paragraph 0052) exerted by the light has been affected (See Tomite Paragraph 0052).
Claim 3:
The combination of Kim, Tomite, and Tsuge discloses wherein the determination unit (See Kim Figure 1; Paragraphs 0022-0023) determines that the light existing in the real space (See Tomite Paragraphs 0016-0018; 0033-0034) exerts an influence on the virtual space (See Tomite Paragraph 0052), if it is determined that the light reaches a display surface of the display apparatus (See Kim Figure 1; Paragraphs 0022-0023).
Claim 4:
The combination of Kim, Tomite, and Tsuge discloses wherein the generation unit (See Kim Figure 2; Paragraphs 0022-0023) generates the image in which the influence (See Tomite Paragraph 0052) exerted by the light existing in the real space (See Tomite Paragraphs 0016-0018; 0033-0034) has been affected (See Kim Paragraphs 0022-0023), by changing a brightness (See Kim Paragraph 0082) and a saturation of a virtual region of the image lighted up and influenced by the light (See Kim Paragraph 0082).
Claim 5:
The combination of Kim, Tomite, and Tsuge discloses wherein the generation unit (See Kim Figure 2; Paragraphs 0022-0023) generates the image (See Kim Figure 2; Paragraphs 0022-002310) by rendering the three-dimensional model (See Kim Paragraphs 0022-002311) of the virtual space using a viewpoint (See Kim Paragraph 002212) of the image capture apparatus (See Kim Paragraphs 0023-0024; 003213), and the generation unit (See Kim Figure 2; Paragraphs 0022-002314) generates the image (See Kim Paragraphs 0023-0024; 003215) in which the influence (See Tomite Paragraph 0052) exerted by the light (See Kim Paragraphs 0012; 0022-002316) existing in the real space (See Tomite Paragraphs 0016-0018; 0033-0034) on the virtual space has been affected (See Kim Paragraph 002217), by mapping the light source (See Kim Paragraphs 0022-0023) onto the virtual space (See Kim Paragraph 002218) and rendering the three-dimensional model (See Kim Paragraphs 0022-002319).
Claim 6:
The combination of Kim, Tomite, and Tsuge discloses wherein the influence (See Tomite Paragraph 0052) includes shade and shadow (See Tomite Paragraphs 0034-0036) that occurs in a virtual object present in the virtual space due to the light source (See Kim Paragraphs 0012; 0022-002320).
Claim 7:
The combination of Kim, Tomite, and Tsuge discloses wherein the influence includes shadow (See Tomite Paragraphs 0034-0036) produced by a real object due to the light source (See Kim Paragraphs 0022-0023).
Claim 8:
The combination of Kim, Tomite, and Tsuge discloses wherein the information relating to the light source (See Kim Paragraphs 0022-0023) includes, out of a three-dimensional position of the light source (See Kim Paragraphs 0022-0023), an orientation or irradiation direction of the light source (See Kim Paragraphs 0022-0023), a brightness of the light source (See Kim Paragraphs 0022-0023), and a type of the light source (See Kim Paragraphs 0022-0023), at least the three-dimensional position (See Kim Paragraphs 0022-0023).
Claim 9:
The combination of Kim, Tomite, and Tsuge discloses wherein the acquisition unit (See Kim Figure 1; Paragraphs 0022-0023) acquires the three-dimensional position of the light source (See Kim Paragraphs 0022-0023), based on an image captured of the real space by a different image capture apparatus from the image capture apparatus (See Kim Paragraphs 0023-0024; 003221).
Claim 10:
The combination of Kim, Tomite, and Tsuge discloses wherein the acquisition unit acquires the three-dimensional position (See Kim Figure 1; Paragraphs 0022-0023) and the orientation of the light source (See Kim Paragraphs 0022-0023), based on an output of a sensor (See Tomite Paragraph 0043) provided in the light source (See Kim Paragraphs 0022-0023).
Claim 11:
The combination of Kim, Tomite, and Tsuge discloses wherein the acquisition unit (See Kim Figure 1; Paragraphs 0022-0023) acquires the information relating to the light source (See Kim Paragraphs 0022-0023) from a control apparatus (See Tomite Paragraph 0071) configured to control an operation of the light source (See Kim Paragraphs 0022-0023).
Claim 12:
The combination of Kim, Tomite, and Tsuge discloses wherein at least some of the information relating to the light source (See Kim Paragraphs 0022-0023) is stored in advance (See Kim Paragraph 0057).
Claims 13-15:
Claims 13-15 are rejected on the same basis as claim 1.
Claim 16:
The combination of Kim, Tomite, and Tsuge discloses wherein the acquired information relating to the irradiation range (See Tsuge Paragraphs 0149; 0172; 0213; 0584) of the light source existing in real space (See Kim Paragraphs 0022-0023) includes information on at least one light source that a real object autonomously controls or at least one light source that dynamically changes (See Kim Paragraphs 0008-0012).
Claim 17:
The combination of Kim, Tomite, and Tsuge discloses wherein the acquired information relating to the irradiation range (See Tsuge Paragraphs 0149; 0172; 0213; 0584) of the light source existing in real space (See Kim Paragraphs 0022-0023) includes information on at least one light source that a real object autonomously controls or at least one light source that dynamically changes (See Kim Paragraphs 0008-0012).
Claim 18:
The combination of Kim, Tomite, and Tsuge discloses wherein the generation unit calculates, for each pixel of the background image (See Tomite Paragraphs 0055-0058), an amount of variation in saturation (See Tsuge Paragraph 0149) and luminance as the influence exerted by the real light source (See Kim Paragraphs 0022-0023), wherein said influence is based on a luminance distribution (See Tsuge Paragraph 0584) in the region of the virtual display surface intersecting the irradiation range (See Tsuge Paragraphs 0169; 0172) of the real light source (See Kim Paragraphs 0022-0023) and based on a color temperature (See Tsuge Paragraph 0213) of the real light source (See Kim Paragraphs 0022-0023).
Claim 19:
The combination of Kim, Tomite, and Tsuge discloses calculating, for each pixel of the background image (See Tomite Paragraphs 0055-0058), an amount of variation in saturation and luminance as the influence exerted by the real light source (See Kim Paragraphs 0022-0023), based on a luminance distribution in the region of the virtual display surface intersecting the irradiation range (See Tsuge Paragraphs 0169; 0172) of the real light source (See Kim Paragraphs 0022-0023) and based on a color temperature (See Tsuge Paragraph 0213) of the real light source (See Kim Paragraphs 0022-0023).
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US Patent No. 7468778 discloses a method of providing a visual representation of a scene to a participant in a studio is provided, wherein the studio is arranged to be viewed by a camera to provide a camera image and has at least a portion of a background object arranged to be keyed out of the camera image by a keying process. The method includes obtaining an estimate of the viewpoint of the participant; rendering an image of the scene based on the viewpoint of the participant; and projecting at least a portion of the rendered image onto the background object to provide a projected image portion visible to the participant. The projected image portion is projected so as to be keyed out of the camera image by the keying process.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHEREE N BROWN whose telephone number is (571)272-4229. The examiner can normally be reached M-F 5:30-2:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, SAID BROOME can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHEREE N BROWN/Primary Examiner, Art Unit 2612 July 1, 2026
1 Paragraph 0074 of Kim recites “A content object generator generates a content object model for acquiring 3D content with respect to a plurality of images acquired according to the setting result.”
2 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform” and “a light source estimated based on an image used to generate the 2.5D background model”.
3 Paragraph 0032 of Kim recites “2D/3D camera”.
4 Paragraph 0074 of Kim recites “A content object generator generates a content object model for acquiring 3D content with respect to a plurality of images acquired according to the setting result.”
5 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform” and “a light source estimated based on an image used to generate the 2.5D background model”.
6 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform.”
7 Paragraph 0074 of Kim recites “A content object generator generates a content object model for acquiring 3D content with respect to a plurality of images acquired according to the setting result.”
8 Paragraph 0052 of Tomite recites “selects and extracts a limited number of light sources which may have significant influence.”
9 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform” and “a light source estimated based on an image used to generate the 2.5D background model”.
10 Paragraph 0074 of Kim recites “A content object generator generates a content object model for acquiring 3D content with respect to a plurality of images acquired according to the setting result.”
11 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform” and “a light source estimated based on an image used to generate the 2.5D background model”.
12 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform.”
13 Paragraph 0032 of Kim recites “2D/3D camera”.
14 Paragraph 0074 of Kim recites “A content object generator generates a content object model for acquiring 3D content with respect to a plurality of images acquired according to the setting result.”
15 Paragraph 0032 of Kim recites “2D/3D camera”.
16 Paragraph 0012 of Kim recites “automatically extracting a moving object and implementing AR, and a more realistic image, and realistic content can be generated by using a rendering scheme by means of simple light source estimation.”
17 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform.”
18 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform.”
19 Paragraph 0022 of Kim recites “a 3-Dimensional (3D) virtual studio platform” and “a light source estimated based on an image used to generate the 2.5D background model”.
20 Paragraph 0012 of Kim recites “automatically extracting a moving object and implementing AR, and a more realistic image, and realistic content can be generated by using a rendering scheme by means of simple light source estimation.”
21 Paragraph 0032 of Kim recites “2D/3D camera”.
Prosecution Insights