DETAILED ACTION
Response to Arguments
Applicant’s arguments with respect to claims 1-21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/18/2026 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 06/04/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 9, 12-16 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) and in view of Zink et al. (herein after will be referred to as Zink) (US 20220382502).
Regarding claim 1, Swatko discloses an information processing device, comprising:
circuitry configured to perform, on a captured video obtained by capturing a display video displayed on a display device and an object separate from the display device, at least one of video processing of a display video area determined using mask information for separating a display video and an object video in the captured video, or video processing of an object video area determined using the mask information. [See Swatko [Figs. 1 and 9] Camera (41) with PC (43) for capturing a user against a LED background. Also, see Fig. 10, Luma mask. Also, see Col. 2 lines 2-3, background/foreground images in both still and motion picture photography and/or videography.]
Swatko does not explicitly disclose
wherein the circuitry performs the at least one of the video processing of the display video or the video processing of the object video according to at least one of a spatial relationship between the display device and a camera configured to obtain the captured video or a spatial relationship between the display device and the object.
However, Zink does disclose
wherein the circuitry performs the at least one of the video processing of the display video or the video processing of the object video according to at least one of a spatial relationship between the display device and a camera configured to obtain the captured video or a spatial relationship between the display device and the object. [See Zink [0038 and Fig. 1] Post processing of video for video recording of a user in front of a LED wall. Also, see 0039, using camera position and orientation for image processing. Also, see 0044-0045, calculating color attenuation factors using an offset between the camera and the wall.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko to add the teachings of Zink, in order to improve upon video content in virtual production [See Zink [0005]].
Regarding claim 3, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein the circuitry performs processing of reducing artifacts as the video processing of the display video area in the captured video. [See Swatko [Col. 3 line 28] Remove visual artifacts from the composite digital image.]
Regarding claim 9, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein the circuitry performs video correction processing of the display video area as the video processing of the display video area in the captured video. [See Swatko [Col. 3 line 28] Remove visual artifacts from the composite digital image.]
Regarding claim 12, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein the circuitry performs video correction processing of the object video area as the video processing of the object video area in the captured video. [See Swatko [Col. 3 line 28] Remove visual artifacts from the composite digital image.]
Regarding claim 13, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein at time of imaging, the circuitry performs the video processing of the display video area or the video processing of the object video area for each frame of the captured video. [See Swatko [Fig. 20]
Regarding claim 14, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein at time of imaging, the circuitry generates the mask information for each frame of the captured video, and determines the display video area and the object video area in the frame. [See Swatko [Fig. 20]
Regarding claim 15, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein the circuitry reads each frame of the captured video from a recording medium, reads mask information recorded corresponding to each frame from the recording medium, and performs the video processing of the display video area or the video processing of the object video area for each frame of the captured video. [See Swatko [Fig. 20]
Regarding claim 16, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein the circuitry reads imaging information corresponding to each frame of the captured video from the recording medium, determines a frame to be a video processing target on a basis of the imaging information, and performs the video processing of the display video area or the video processing of the object video area for the frame determined to be the video processing target. [See Swatko [Fig. 20]
Regarding claim 19, see examiners rejection for claim 1 which is analogous and applicable for the rejection of claim 19.
Regarding claim 20, see examiners rejection for claim 1 which is analogous and applicable for the rejection of claim 20.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) in view of Zink (US 20220382502) and in further view of Dsouza et al. (herein after will be referred to as Dsouza) (US 20190355172).
Regarding claim 2, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko discloses
wherein the display video displayed on the display device includes a background video obtained by rendering[See Swatko [Figs. 1 and 9] Camera (41) with PC (43) for capturing a user against a LED background. Also, see Fig. 10, Luma mask. Also, see Col. 2 lines 2-3, background/foreground images in both still and motion picture photography and/or videography.]
Swatko does not explicitly disclose
However, Dsouza does disclose
[See Dsouza [0055] Background is 3D.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink) to add the teachings of Dsouza, in order to perform a simple substitution of 2D image data with 3D image data. Furthermore, Dsouza assists in improving upon replacing background portions [See Dsouza [0002]].
Claims 4 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) in view of Zink (US 20220382502) and in further view of Cordes et al. (herein after will be referred to as Cordes) (US 20200145644).
Regarding claim 4, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs moire reduction processing as the video processing of the display video area in the captured video.
However, Cordes does disclose
wherein the circuitry performs moire reduction processing as the video processing of the display video area in the captured video. [See Cordes [0114] Moire patterns occur with the use of video display screens. Methods to alleviate the moire effect include editing the image post production.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink) to add the teachings of Cordes, in order to remove unwanted effects due to moire. Cordes discloses that the use of video display screens in digital videography causes moire [See Cordes [0114]].
Regarding claim 10, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs moire reduction processing as the video processing of the object video area in the captured video.
However, Cordes does disclose
wherein the circuitry performs moire reduction processing as the video processing of the object video area in the captured video. [See Cordes [0114] Moire patterns occur with the use of video display screens. Methods to alleviate the moire effect include editing the image post production or modifications to the display.]
Applying the same motivation as applied in claim 4.
Claims 5-8 are rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) in view of Zink (US 20220382502and in further view of Hsiao et al. (herein after will be referred to as Hsiao) (US 20210201449).
Regarding claim 5, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs moire generation degree determination in the display video area, and performs moire reduction processing according on a determination result, as the video processing of the display video area in the captured video.
However, Hsiao does disclose
wherein the circuitry performs moire generation degree determination in the display video area, and performs moire reduction processing according on a determination result, as the video processing of the display video area in the captured video. [See Hsiao [0035] Moire pattern determination unit examines Moire pattern and performs color correction upon pixels that are determined to be part of the Moire pattern.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink) to add the teachings of Hsiao, in order to remove unwanted effects due to moire. It is obvious/inherent that moire effects are generated when imaging a video display screen.
Regarding claim 6, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs moire generation degree determination in the display video area, sets processing intensity according to a determination result, and performs moire reduction processing, as the video processing of the display video area in the captured video.
However, Hsiao does disclose
wherein the circuitry performs moire generation degree determination in the display video area, sets processing intensity according to a determination result, and performs moire reduction processing, as the video processing of the display video area in the captured video. [See Hsiao [0035] Moire pattern determination unit examines Moire pattern and performs color correction upon pixels that are determined to be part of the Moire pattern….the colors are corrected according to different rates based on different levels of the Moire.]
Applying the same motivation as applied in claim 5.
Regarding claim 7, Swatko (modified by Zink and Hsiao) disclose the device of claim 5. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs the moire generation degree determination by comparing the captured video with the display video.
However, Hsiao does disclose
wherein the circuitry performs the moire generation degree determination by comparing the captured video with the display video. [See Hsiao [0035] Moire pattern determination unit examines Moire pattern and performs color correction upon pixels that are determined to be part of the Moire pattern. Also, see 0005, when photographing a display screen.]
Applying the same motivation as applied in claim 5.
Regarding claim 8, Swatko (modified by Zink and Hsiao) disclose the device of claim 5. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs the moire generation degree determination based on imaging information of a camera at time of imaging or imaging environment information of an imaging facility.
However, Hsiao does disclose
wherein the circuitry performs the moire generation degree determination based on imaging information of a camera at time of imaging or imaging environment information of an imaging facility. [See Hsiao [0035] Moire pattern determination unit examines Moire pattern and performs color correction upon pixels that are determined to be part of the Moire pattern. Also, see 0005, when photographing a display screen.]
Applying the same motivation as applied in claim 5.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) in view of Zink (US 20220382502) and in further view of Hsiao (US 20210201449) and in further view of Sudo et al. (herein after will be referred to as Sudo) (US 20220408037).
Regarding claim 11, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs determination processing for clothing of a subject, and
performs moire reduction processing according to a determination result, as the video processing of the object video area in the captured video.
However, Hsiao does disclose
performs moire reduction processing according to a determination result, as the video processing of the object video area in the captured video. [See Hsiao [0035] Moire pattern determination unit examines Moire pattern and performs color correction upon pixels that are determined to be part of the Moire pattern. Also, see 0005, when photographing a display screen.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink) to add the teachings of Hsiao, in order to remove unwanted effects due to moire. It is obvious/inherent that moire effects are generated when imaging a video display screen.
Swatko (modified by Zink and Hsiao) do not explicitly disclose
wherein the circuitry performs determination processing for clothing of a subject, and
However, Sudo does disclose
wherein the circuitry performs determination processing for clothing of a subject, and [See Sudo [0039] Extracts objects such as human face and clothes based on image data.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink and Hsiao) to add the teachings of Sudo, in order to utilize object recognition for foreground detection. This will assist in the accuracy of foreground detection.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) in view of Zink (US 20220382502) and in further view of Piratla et al. (herein after will be referred to as Piratla) (US 20130236073).
Regarding claim 17, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko does not explicitly disclose
wherein the mask information is generated based on a video obtained by an infrared short wavelength camera configured to capture a same video as the captured video.
However, Piratla does disclose
wherein the mask information is generated based on a video obtained by an infrared short wavelength camera configured to capture a same video as the captured video. [See Piratla [0041] Video camera is a combination of an RGB camera and an infrared camera. Also, see 0030, analyzing IR video for object/background separation.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink) to add the teachings of Sudo, in order to perform a simple substitution of cameras for performing the foreground/background segmentation.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) in view of Zink (US 20220382502) in view of Piratla (US 20130236073) and in further view of Richards et al. (herein after will be referred to as Richards) (US 20230054197).
Regarding claim 18, Swatko (modified by Zink and Piratla) disclose the device of claim 17. Furthermore, Swatko does not explicitly disclose
wherein the infrared short wavelength camera is configured in such a manner that subject light of the infrared short wavelength camera is initially incident on a same optical axis as the camera configured to obtain the captured video obtained by capturing the display video and the object, and the subject light of the infrared short wavelength camera is subsequently separated by a beam splitter.
However, Richards does disclose
wherein the infrared short wavelength camera is configured in such a manner that subject light of the infrared short wavelength camera is initially incident on a same optical axis as the camera configured to obtain the captured video obtained by capturing the display video and the object, and the subject light of the infrared short wavelength camera is subsequently separated by a beam splitter. [See Richards [Fig. 8A] Dual-band visible/IR camera housing with separate imagers and a beamsplitter. The dual-band camera incorporates a single optical axis for both the visible and IR cameras.]
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink and Piratla) to add the teachings of Richards, in order to incorporate obvious imaging optics such as a beamsplitter for optimizing the camera housing for a dual-camera imaging assembly.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Swatko (US Patent No. 9,681,041) and in view of Zink et al. (herein after will be referred to as Zink) (US 20220382502) and in further view of Hirschfield et al. (herein after will be referred to as Hirschfield) (US Patent No. 12,373,999).
Regarding claim 21, Swatko (modified by Zink) disclose the device of claim 1. Furthermore, Swatko does not explicitly disclose
wherein the circuitry performs the at least one of the video processing of the display video or the video processing of the object video further according to a spatial relationship between the display device and the object.
However, Hirschfield does disclose
wherein the circuitry performs the at least one of the video processing of the display video or the video processing of the object video further according to a spatial relationship between the display device and the object. [See Hirschfield [Col. 11 lines 30-45]].
It would have been obvious to the person of ordinary skill in the art at the time of the effective filing date to modify the device by Swatko (modified by Zink) to add the teachings of Hirschfield, in order to improve the images shown on displays for an immersive content production system [See Hirschfield [Col. 1 lines 32-53]].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES T BOYLAN whose telephone number is (571)272-8242. The examiner can normally be reached Monday-Friday 7am-3pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JAMIE ATALA can be reached at 571-272-7384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES T BOYLAN/Examiner, Art Unit 2486
Prosecution Insights