DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 8/4/2026 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9/4/2026 was filed after the mailing date of the claims on 12/23/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 2, 3, 4, 10, 11, 12, 13, 16, 17, 18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claim(s) 1, 2, 3, 4, 10, 11, 12, 13, 16, 17, 18 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent 9,480,907 Benko et al. (hereinafter Benko) in view of U.S. Patent 12052417, Kim et al. (hereinafter Kim).
2. Regarding Claim 1, Benko discloses A first electronic device (Fig.1 102 TV; Col. 2 lines 14-15, “a primary display 102 (e.g., television)”) that provides a first part of an input image (Col. 2 lines 15-16, “a primary image (e.g., primary image 104 of primary display 102)”) in conjunction with a second electronic device projecting a second part of the input image (Fig. 2: 200 projector; Col. 2lines 19-20, “environmental display 200 [i.e. projector] used to display a peripheral illusion in display environment 100”),
the first electronic device (Fig. 1: 102 TV) comprising:
a communication interface (inherent, Fig. 8: 814 communication subsystem);
a display (Col. 2 line 14, “display a primary image on primary display 102”);
a memory storing one or more instructions (Fig. 8: 810 storage); and
at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”) operatively coupled to the communication interface (Fig. 8: 814 communication subsystem; Col. 9 lines8-20, “communication subsystem 814 may be configured to communicatively couple computing system 800 with one or more other computing devices. Communication subsystem 814 may include wired and/or wireless communication devices compatible with one or more different communication protocols”),
the display (Col. 2 line 14, “display a primary image on primary display 102”) and the memory (Fig. 8: 810 storage) and configured to execute the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”) in the memory (Fig. 8: 810 storage), wherein the one or more instructions, when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”),
cause the first electronic device (Fig. 1: 102 TV) to:
obtain information about a projection surface to which the second electronic device projects the second part of the input image (Fig. 2; Col. 2 lines 19-25, “environmental display 200 used to display a peripheral illusion in display environment 100. A peripheral image from the environmental display 200 may be projected as a peripheral illusion around the primary display 102 so that the peripheral illusion appears as an extension of the primary image displayed by the primary display.”),
segment (Col. 4 lines 28-34, “FIG. 4 shows a peripheral illusion 400 in which the game extends only onto the rear wall surrounding the primary display 102. In this non-limiting example, peripheral illusion 400 may be masked from one or more surfaces in the display environment. The one or more masked objects may include the primary display 102, all surfaces except those surfaces within a threshold distance of a projection plane." Fig. 4; Col. 4 lines 39-40, “such as wall 108 of FIG. 1, or any other surface within a threshold distance of a segmented projection plane”), based on a flag included in metadata information in the input image indicating whether the input image is configured to be segmented into different parts (Fig.4; Col. 7 lines 1-15, “Peripheral illusions may optionally be executed by retrieving the rendering information for the peripheral illusions directly from the content defining the primary image.” Col. 7 lines 11-17, “an effects track that accompanies a movie in the same way as a subtitle track could trigger peripheral illusions. Such effects tracks may be delivered to a user by the same or a separate vendor than the original broadcaster/creator.” Metadata included alongside and within the content stream. Col. 2 lines 5-10, “a primary or secondary broadcaster may send additional information to an environmental display that is synchronized with the main broadcast.” Information embedded in or sent alongside the content),
the input image into the first part corresponding to a first area (Col. 2 lines 15-16, “a primary image (e.g., primary image 104 of primary display 102)”) and the second part corresponding to a second area based on the information about the projection surface (Figs. 1-6, Col. 4 lines 39-40, “The wall, such as wall 108 of FIG. 1, or any other surface within a threshold distance of a segmented projection plane.”) and information about the first electronic device (Col. 8 lines 29-30, “output the primary image [i.e., information about display] to the primary display”), and
control (Col. 7 line 8, “controller input”, also lines 40-42) the display to display the first part of the input image based on information about the first area (Col. 2 lines 15-16, “a primary image (e.g., primary image 104 of primary display 102)”), and
control (Col. 7 line 8, “controller input”, also lines 40-42) the communication interface to transmit information (Fig. 8: 814 communication subsystem; Col. 9 lines8-20, “communication subsystem 814 may be configured to communicatively couple computing system 800 with one or more other computing devices. Communication subsystem 814 may include wired and/or wireless communication devices compatible with one or more different communication protocols”) about the second part (Fig. 3: 300; Col. 4 line 16, “The peripheral illusion 300”) corresponding to the second area to the second electronic device (Abstract, “A peripheral illusion is displayed around the primary display by an environmental display so that the peripheral illusion appears as an extension of the primary image.”).
However, Benko does not explicitly disclose segment, based on a flag included in metadata information in the input image indicating whether the input image is configured to be segmented into different parts
Kim teaches segment, based on a flag included in metadata information in the input image indicating whether the input image is configured to be segmented into different parts (Fig. 9, decoding execution unit 910; Col. 11 lines 62-67, “The video encoding apparatus may determine whether a picture is partitioned into one or more sub-units, encode a result (split flag), and signal the encoded result. The decoding execution unit 910 may decode the split flag from a bitstream (S1010) and determine whether a picture is partitioned into sub-units based on the split flag.” Claim 1, “decoding… partition information related to partitioning of the picture into a plurality of slices when the split flag indicates partitioning;” Fig. 10, steps S1010-S1030).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Benko’s system such that the metadata information which optionally triggers/enables segmentation of the input image (Benko, Col. 7 lines 1-15; Col. 2 lines 5-10) includes a flag, as taught by Kim, indicating whether the input image is configured to be segmenting into the first part/first area and second part/second area based on that flag, because Kim teaches that using a discrete flag to indicate whether an image is to be partitioned, checked by the decoder before partition information is retrieved or acted upon, is a known and conventional technique for signaling optional image-partitioning behavior in exactly the technical field (image/video coding and multi-source image synthesis).
3. Regarding Claim 2, Benko discloses The first electronic device of claim 1, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”), cause the first electronic device (Fig. 1: 102 TV) to:
receive, through the communication interface (Col. 9 lines 18-20, “the communication subsystem may allow computing system 800 to send and/or receive messages to and/or from other devices via a network such as the Internet.”), an image about the projection surface photographed by a camera included in the second electronic device (Fig. 2; 204 sensor part of projector 200; Col. 2 lines 51-54, “Sensor 204 may include a depth camera (e.g., structured light, time-of-flight, stereoscopic, or other depth camera), a color camera.”); and
identify an area where the first electronic device is positioned on the projection surface based on the image about the projection surface (Claim 14, “a peripheral input configured to receive a depth input from a depth camera, the depth input identifying a location of the primary display and features of display environment on which the peripheral illusion is projected”).
4. Regarding Claim 3, Benko discloses The first electronic device of claim 2, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”), cause the first electronic device (Fig. 1: 102 TV) to:
identify the area where the first electronic device is positioned (Claim 14, “a peripheral input configured to receive a depth input from a depth camera, the depth input identifying a location of the primary display) on the projection surface of the image about the projection surface as the first area (Fig. 4; See Col. 4 lines 28-35) and
identify other areas excluding the first area in the image about the projection surface as the second area (Fig. 4; Col. 4 lines 28-35, “FIG. 4 shows a peripheral illusion 400 in which the game extends only onto the rear wall surrounding the primary display 102. In this non-limiting example, peripheral illusion 400 may be masked from one or more surfaces in the display environment. The one or more masked objects may include the primary display 102, all surfaces except those surfaces within a threshold distance of a projection plane (e.g., a wall behind the primary display)”).
5. Regarding Claim 4, Benko discloses The first electronic device of claim 1, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”), cause the first electronic device to:
receive, through the communication interface (Fig. 8: 814 communication subsystem; Col. 9 Lines 18-20, “the communication subsystem may allow computing system 800 to send and/or receive messages to and/or from other devices via a network such as the Internet.”), information about a distance between the second electronic device and the projection surface via a distance sensor included in the second electronic device (Fig. 2: 204; Col. 2 Lines 51-53, “Sensor 204 may include a depth camera (e.g., structured light, time-of-flight, stereoscopic, or other depth camera), a color camera.” Col. 7 Lines 35-38, “the computing system may be integrated with a stand-alone environmental display (e.g., projector) including a depth camera and/or other sensors”),
obtain information about a size and a location of the projection surface (Fig. 6; Col. 5 lines 1-8, “The position and size of the individual dots may be moved/changed to create the illusion of apparent 3D motion.”) based on the information about the distance between the second electronic device and the projection surface (Fig. 3; Col. 4 Lines 19-21, “the peripheral illusion increases the effective screen size of the primary display by extending images from the primary display out into the rest of the room.”), and segment the input image (Col. 4 lines 39-40, “such as wall 108 of FIG. 1, or any other surface within a threshold distance of a segmented projection plane”) into the first part corresponding to the first area (Col. 2 lines 15-16, “a primary image (e.g., primary image 104 of primary display 102)”) and the second part corresponding the second area (Fig. 3: 300; Col. 4 line 16, “The peripheral illusion 300”) based on the information about the size and the location of the projection surface (Fig. 6; Col. 5 lines 1-8, “The position and size of the individual dots may be moved/changed to create the illusion of apparent 3D motion.”).
6. Regarding Claim 10, Benko discloses The first electronic device of claim 1, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”), cause the first electronic device to:
operate in one of a first mode which displays the input image in the first electronic device (Col. 2 lines 15-16, “a primary image (e.g., primary image 104 of primary display 102)”) and a second mode which segments (Fig. 4; Col. 4 lines 39-40, “such as wall 108 of FIG. 1, or any other surface within a threshold distance of a segmented projection plane”) and displays the input image through the first electronic device and the second electronic device (Col. Lines, 47-53, “FIG. 5 shows a peripheral illusion 500 in which only certain game elements escape the primary image 502. For instance, in the illustrated first-person shooter, only explosion 506 is bled out of the primary image 502. Therefore, peripheral illusions may include an extension of selected elements of the primary image while not extending other elements (e.g., building 504).”).
7. Claim 11 is a method claim, rejected with respect to the same limitation rejected in device claim 1.
8. Regarding Claim 12, Benko discloses The method of claim 11, wherein the obtaining the information about the projection surface further comprises obtaining an image about the projection surface photographed by a camera (Fig. 2: 204; Col. 2 lines 51-54, “Sensor 204 may include a depth camera (e.g., structured light, time-of-flight, stereoscopic, or other depth camera), a color camera.”) included in the second electronic device (Fig. 200 environmental display; Col. 7 lines 36-38, “stand-alone environmental display (e.g., projector) including a depth camera and/or other sensor”); and
wherein the method further comprises identifying an area where the first electronic device is positioned on the projection surface based on the image about the obtained projection surface (Claim 14, “a peripheral input configured to receive a depth input from a depth camera, the depth input identifying a location of the primary display and features of display environment on which the peripheral illusion is projected”).
9. Regarding Claim 13, Benko discloses The method of claim 12, further comprising:
identifying the area where the first electronic device is positioned on the projection surface of the obtained image as the first area and identifying other areas excluding the first area in the obtained image as the second area (Fig. 4; Col. 4 lines 28-35, “FIG. 4 shows a peripheral illusion 400 in which the game extends only onto the rear wall surrounding the primary display 102. In this non-limiting example, peripheral illusion 400 may be masked from one or more surfaces in the display environment. The one or more masked objects may include the primary display 102, all surfaces except those surfaces within a threshold distance of a projection plane (e.g., a wall behind the primary display)”).
10. Claim 16 is a non-transitory CRM claim, rejected with respect to the same limitation rejected in device claim 1.
11. Regarding Claim 17, Benko discloses The non-transitory computer readable medium of claim 16, wherein the obtaining the information about the projection surface further comprises obtaining an image about the projection surface photographed by a camera (Fig. 2: 204; Col. 2 lines 51-54, “Sensor 204 may include a depth camera (e.g., structured light, time-of-flight, stereoscopic, or other depth camera), a color camera.”) included in the second electronic device (Fig. 200 environmental display; Col. 7 lines 36-38, “stand-alone environmental display (e.g., projector) including a depth camera and/or other sensor”); and
wherein the method further comprises identifying an area where the first electronic device is positioned on the projection surface based on the image about the obtained projection surface (Claim 14, “a peripheral input configured to receive a depth input from a depth camera, the depth input identifying a location of the primary display and features of display environment on which the peripheral illusion is projected”).
12. Regarding Claim 18, Benko discloses The non-transitory computer readable medium of claim 17, wherein the method further comprises:
identifying the area where the first electronic device is positioned on the projection surface of the obtained image as the first area and identifying the other areas excluding the first area in the obtained image as the second area (Fig. 4; Col. 4 lines 28-35, “FIG. 4 shows a peripheral illusion 400 in which the game extends only onto the rear wall surrounding the primary display 102. In this non-limiting example, peripheral illusion 400 may be masked from one or more surfaces in the display environment. The one or more masked objects may include the primary display 102, all surfaces except those surfaces within a threshold distance of a projection plane (e.g., a wall behind the primary display)”).
Allowable Subject Matter
Claims 5-9, 14, 15, 19, and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 5, Benko discloses The first electronic device of claim 1, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”),
Benko teaches (Col. 6 lines, “The audio of a game and/or movie may also be affected by events occurring in the primary image and/or peripheral illusion. For example, a virtual object may hit a physical object through the peripheral illusion and a corresponding explosion may sound. In another non-limiting example, a gunshot may echo in the room, and the viewer may hear the sound bounce from one side of the room to another.”).
Benko does not explicitly disclose cause the first electronic device to:
convert a channel of an audio corresponding to the input image to a channel corresponding to an output specification of the second electronic device and an output specification of the first electronic device, and
map the audio to the converted channel based on information about a location of the second electronic device
Regarding Claim 6, Benko discloses The first electronic device of claim 5, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”),
Benko does not explicitly disclose cause the first electronic device to:
analyze the second part of the input image projected to the projection surface and perform post-processing on the audio per the converted channel.
Regarding Claim 7, Benko discloses The first electronic device of claim 1, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”),
Benko does not explicitly disclose cause the first electronic device to: control the display to display a screen of a preset color in the first electronic device and control the communication interface to transmit,
to the second electronic device, a control command to project a screen of the preset color, receive, from the second electronic device through the communication interface, an image capturing the screen of the preset color displayed by the first electronic device and the screen of the preset color projected by the second electronic device, and control brightness of at least one of a screen output by the first electronic device or a screen projected by the second electronic device based on the image capturing the screen of the preset color received from the second electronic device.
Regarding Claim 8, Benko discloses The first electronic device of claim 1, wherein the one or more instructions (Claim 1, “a storage machine holding instructions executable by the logic machine”), when executed by the at least one processor (Fig. 8: 808 logic machine; Col. 8 line 10, “logic machine may include one or more processors”),
Benko does not explicitly disclose cause the first electronic device to:
control the display to display a screen which changes to a pattern preset in the first electronic device and control the communication interface to transmit, to the second electronic device,
a control command to project a screen which changes to a preset pattern,
receive, from the second electronic device through the communication interface,
an image capturing the screen changing to the preset pattern displayed by the first electronic device and the screen changing to the preset pattern projected by the second electronic device, and control at least one of an image output timing of the first electronic device or an image projecting timing of the second electronic device based on the image received from the second electronic device.
Regarding Claim 9, Benko discloses The first electronic device of claim 1,
Benko does not explicitly disclose wherein the flag indicates whether the input image is configured to be segmented into the first part and the second part; and
wherein the one or more instructions, when executed by the at least one processor, cause the first electronic device to:
determine whether to segment the image into the first part corresponding to the first area and the second part corresponding to the second area based on the flag.
Regarding Claim 14, Benko discloses The method of claim 11,
Benko does not explicitly disclose wherein the obtaining the information about the projection surface further comprises:
obtaining information about a distance between the second electronic device and the projection surface through a distance sensor included in the second electronic device; and
obtaining information about a size and a location of the projection surface based on information about the distance between the second electronic device and the projection surface,
wherein the segmenting further comprises segmenting the input image into the first part corresponding to the first area and the second part corresponding to the second area based on the information about the size and the location of the projection surface.
Regarding Claim 15, Benko discloses The method of claim 11, comprising:
Benko does not explicitly disclose converting a channel of an audio corresponding to the input image to a channel corresponding to an output specification of the second electronic device and an output specification of the first electronic device; and
mapping the audio to the converted channel based on information about a location of the second electronic device.
Regarding Claim 19, Benko discloses The non-transitory computer readable medium of claim 16,
Benko does not explicitly disclose wherein the obtaining the information about the projection surface further comprises:
obtaining information about a distance between the second electronic device and the projection surface through a distance sensor included in the second electronic device; and
obtaining information about a size and a location of the projection surface based on information about the distance between the second electronic device and the projection surface,
wherein the segmenting further comprises segmenting the input image into the first part corresponding to the first area and the second part corresponding to the second area based on the information about the size and the location of the projection surface.
Regarding Claim 20, Benko discloses The non-transitory computer readable medium of claim 16,
Benko does not explicitly disclose wherein the method further comprises:
converting a channel of an audio corresponding to the input image to a channel corresponding to an output specification of the second electronic device and an output specification of the first electronic device; and
mapping the audio to the converted channel based on information about a location of the second electronic device.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OMER KHALID whose telephone number is (571)270-5997. The examiner can normally be reached Monday- Friday 9am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Miller can be reached at (571) 272-7353. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OMER KHALID/Examiner, Art Unit 2422
/JOHN W MILLER/Supervisory Patent Examiner, Art Unit 2422
Prosecution Insights