DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/16/2024 AND 10/2/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
3. Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter recited in 35 U.S.C. 101 (process, machine, manufacture, or composition of matter). The Office has released the Interim Examination Instructions for Evaluating Subject Matter Eligibility Under 35 U.S.C. 101, August 24, 2009; p.2 and Subject Matter Eligibility of Computer Readable Media. See 1351 OG 212 February 23, 2010. Claim 14 is directed to “a program….” The applicant’s specification does not specifically define a computer program product within a computer usable medium. For example, applicant’s originally filed specification, at page 27 states “A CPU 71 of the information processing device 70 illustrated in Fig. 8 executes various kinds of processing in accordance with a program stored in a nonvolatile memory unit 74 such as a ROM 72 or, for 5 example, an electrically erasable programmable read-only memory (EEP-ROM), or a program loaded from a storage unit 79 to a RAM 73.” See also original specification at page 98, Therefore, the broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). In an effort to assist the patent community in overcoming a rejection or potential rejection under 35 U.S.C. § 101 in this situation, the USPTO suggests the following approach. A claim drawn to such a computer readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation "non-transitory" to the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
4. Claim(s) 1-4, 9-10, 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Geissler; Michael et al. US 20230336870 A1 and in further view of Rotscholl; Ingo et al. US 20230281758 A1 (hereafter Rotscholl) and in further view of Hirotsune; Satoshi US 20190306384 A1 (hereafter Hirotsune) and in further view of Imoto; Masayoshi et al. US 20150271409 A1 (Imoto).
Regarding claim 1, “an information processing device comprising: a determination unit that performs determination processing of determining whether or not an artifact occurs in a captured video of a camera by using an interval between light emitting elements of a display, a pixel interval between image sensors of the camera, characteristics of an optical low-pass filter of the camera, a focal distance, an F value, and a focusing distance of a lens of the camera, and a distance between the display and the camera in an imaging system that captures a video of the display with the camera” Geissler para 22-25 and 28-35 teaches devices comprising a link device and components of a camera to determine when artifacts occur and for mitigating interference patterns (i.e., artifacts) produced when a camera sensor captures images or video including a display screen used as a background. See para 35 screen is one that can adaptively display information, and that information can be captured by the camera during filming wherein the information may be displayed by being emitted from light emitting devices on the screen (as in the case of an LED wall) or by being projected on to and reflected from the wall. See para 28 disclosing the claimed intervals for a display and sensors of a camera (i.e., the apparent spacing of the grid of pixels on the display screen and pixels on an image sensor of the camera); see also para 20-27 disclosing camera depth of field, focal distance as the distance from the camera to the focused objects in the scene; and distance from the camera to the display screen. With respect to the claimed “F value”, Geissler para 12 and 27 teaches an aperture value that a person of ordinary skill in the art would understand the aperture value is measured in f-stops and takes focal length into consideration. Geissler does not reference the optical low-pass filter of the camera as claimed.
In an analogous art, Rotscholl recognizes a known problem and solutions for a reduction of Moiré interference similar to defocusing can also be achieved by smoothing filtering of a camera image wherein low-pass filters for filtering the camera image in position space are known. See Rotscholl para 7-14 and para 44-52.
In an analogous art, Hirotsune also supports the disclosure of Rotscholl when drawing inferences regarding the use of low-pass filters to implement solutions relating to captured Moire when capturing an image of a display panel that displays a predetermined image using an capture device (camera) and wherein moire is included in the captured image (para 5-8, 43-46).
In an analogous art, Imoto teaches an early motivation for modifying the teachings of Geissler, Rotscholl, and Hirotsune wherein Imoto teaches a moire frequency control unit that performs a control to position the display panel and the camera at a predetermined relative angle with respect to each other. See Imoto Abstract, para 44-52 disclosing utilizing a low-pass filter wherein moire frequency control unit 32 measures the spatial frequency of a moire pattern, and controls the camera 20 so the camera 20 rotates and moves forward and backward, thereby changing the relative angle and relative distance between the display panel 10 and the camera 20. When the relative angle and relative distance between the display panel 10 and the camera 20 are changed, the spatial frequency of a moire pattern that appears in a captured image of the display panel 10 shifts. The moire frequency control unit 32 rotates the camera 20 by a predetermined angle before measuring unevenness, thereby shifting only moire components of the captured image to a relatively high frequency region).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify Geissler’s invention for devices and components comprising a camera to determine when artifacts occur and for mitigating interference patterns (i.e., artifacts) produced when a camera sensor captures images or video including a display screen used as a background by further incorporating known elements of Rotscholl which recognizes a known problem and solutions for a reduction of Moiré interference by smoothing filtering of a camera image wherein low-pass filters for filtering the camera image in position space are used and wherein Hirotsune supports the disclosure of Rotscholl for drawing inferences regarding the use of low-pass filters to implement solutions relating to captured Moire when capturing an image of a display panel that displays a predetermined image using an capture device (camera) and wherein moire is included in the captured image because Imoto identifies a problem relating to interference patterns (moire) produced when a camera sensor captures images or video including a display screen frequency control unit that performs a control to position the display panel and the camera at a predetermined relative angle with respect to each other and further utilizes a low-pass filter and a moire frequency control unit measures the spatial frequency of a moire pattern, and controls the camera so the camera rotates and moves forward and backward in order to alleviate the problem associated with the produced interference patterns (artifacts/moire).
Regarding claim 2, “wherein in the determination processing, the determination unit obtains an amplitude of an image of the light emitting element on an image sensor, and compares the obtained amplitude with a threshold to determine whether an artifact has occurred” is further rejected as discussed in the rejection of claim 1 wherein Hirotsune Fig. 5a, 5c, 8, 11a-b, 12a-c and para 61, 71-75, 79, 109-111 intensity/amplitude and thresholds for determining when moire occurs; see also Imoto para 52, 73-74, 81-82 using panel amplitudes to determine when and where moire patterns are located.
Regarding claim 3, “wherein the determination unit acquires a shortest distance and a longest distance between the display and the camera, and performs the determination processing using the shortest distance and the longest distance” is further rejected as discussed in the rejection of claims 1-2 wherein Geissler (Abstract, para 20-24, 24 determining distance between camera and display in order to implement a focus limit or cap). See also Hirotsune para 48. See also Imoto para 12, 18, 24, 48, 51, 66-72, moire frequency control unit 32 measures the spatial frequency of a moire pattern, and controls the camera 20 so the camera 20 rotates and moves forward and backward, thereby changing the relative angle and relative distance between the display panel 10 and the camera 20. See also Imoto para 80 if the distance (relative distance) between the camera and the display panel is changed to enlarge or reduce a video, the relationship between the Nyquist frequency and the frequency of a moire pattern changes as a result of the change in distance, so that a change occurs in the interval between each of the moire pattern frequencies that are indicated by the symbols x equally spaced in FIG. 8.
Regarding claim 4, “wherein in the determination processing, the determination unit determines a range in which an artifact occurs in the captured video” is further rejected as discussed in the rejection of claims 1-3 wherein Geissler (Abstract, para 20-24, 24 determining distance between camera and display in order to implement a focus limit or cap). See also Hirotsune para 48 and Hirotsune Fig. 5a, 5c, 8, 11a-b, 12a-c and para 61, 71-75, 79, 109-111 intensity/amplitude and thresholds for determining when moire occurs; see also Imoto para 52, 73-74, 81-82 using panel amplitudes to determine when and where moire patterns are located. See also Imoto para 12, 18, 24, 48, 51, 66-72, moire frequency control unit 32 measures the spatial frequency of a moire pattern, and controls the camera 20 so the camera 20 rotates and moves forward and backward, thereby changing the relative angle and relative distance between the display panel 10 and the camera 20. See also Imoto para 80 if the distance (relative distance) between the camera and the display panel is changed to enlarge or reduce a video, the relationship between the Nyquist frequency and the frequency of a moire pattern changes as a result of the change in distance, so that a change occurs in the interval between each of the moire pattern frequencies that are indicated by the symbols x equally spaced in FIG. 8.
Regarding claim 9, “wherein the determination unit acquires all or some of values as an interval between light emitting elements of the display, a pixel interval between image sensors of the camera, characteristics of an optical low-pass filter of the camera, a focal distance of a lens of the camera, an F value, a focusing distance, and a distance between the display and the camera, on the basis of information received by communication with another device” is further rejected as discussed in the rejection of claims 1-4 wherein Geissler (Abstract, para 20-24, 24 determining distance between camera and display in order to implement a focus limit or cap; see also Fig. 1 elements 1, 10, 6, 8, 9 in communication to providing all or some values as claimed). See also Hirotsune para 48 and Hirotsune Fig. 5a, 5c, 8, 11a-b, 12a-c and para 61, 71-75, 79, 109-111 intensity/amplitude and thresholds for determining when moire occurs; see also Imoto para 52, 73-74, 81-82 using panel amplitudes to determine when and where moire patterns are located. See also Imoto para 12, 18, 24, 48, 51, 66-72, moire frequency control unit 32 measures the spatial frequency of a moire pattern, and controls the camera 20 so the camera 20 rotates and moves forward and backward, thereby changing the relative angle and relative distance between the display panel 10 and the camera 20. See also Imoto para 80 if the distance (relative distance) between the camera and the display panel is changed to enlarge or reduce a video, the relationship between the Nyquist frequency and the frequency of a moire pattern changes as a result of the change in distance, so that a change occurs in the interval between each of the moire pattern frequencies that are indicated by the symbols x equally spaced in FIG. 8.
Regarding claim 10, “wherein the determination unit is configured to: acquire a shortest distance and a longest distance between the display and the camera; and compare a maximum value of an amplitude of an image of the light emitting element on an image sensor with a threshold in a plurality of distance values from a shortest distance to a longest distance, and determine that an artifact has occurred in a case where the maximum value exceeds the threshold” is further rejected as discussed in the rejection of claims 1-4, 8 wherein Geissler (Abstract, para 20-24, 24 determining distance between camera and display in order to implement a focus limit or cap). See also Hirotsune para 48 and Hirotsune Fig. 5a, 5c, 8, 11a-b, 12a-c and para 61, 71-75, 79, 109-111 intensity/amplitude and thresholds for determining when moire occurs; see also Imoto para 52, 73-74, 81-82 using panel amplitudes to determine when and where moire patterns are located. See also Imoto para 12, 18, 24, 48, 51, 66-72, moire frequency control unit 32 measures the spatial frequency of a moire pattern, and controls the camera 20 so the camera 20 rotates and moves forward and backward, thereby changing the relative angle and relative distance between the display panel 10 and the camera 20. See also Imoto para 80 if the distance (relative distance) between the camera and the display panel is changed to enlarge or reduce a video, the relationship between the Nyquist frequency and the frequency of a moire pattern changes as a result of the change in distance, so that a change occurs in the interval between each of the moire pattern frequencies that are indicated by the symbols x equally spaced in FIG. 8.
Regarding the method claim 13 and the program claim 14 are grouped and rejected with the device claims 1-4, 9-10 because the steps of the method claims are met by the disclosure of the apparatus and methods of the reference(s) as discussed in the rejection of claims 1-4, 9-10 and because the elements of a device are easily converted into steps of a method or elements of computer implemented methods by one of ordinary skill in the art.
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Geissler; Michael et al. US 20230336870 A1 and in further view of Rotscholl; Ingo et al. US 20230281758 A1 (hereafter Rotscholl) and in further view of Hirotsune; Satoshi US 20190306384 A1 (hereafter Hirotsune) and in further view of Imoto; Masayoshi et al. US 20150271409 A1 (Imoto) and in further view of Tsuda; Shinsuke et al. US 20200004481 A1 (hereafter Tsuda).
Regarding claim 5, “wherein the determination unit performs warning processing in response to determination that an artifact has occurred” the combination of Geissler, Rotscholl, Hirotsune, and Imoto are silent with respect to warning as claimed. In an analogous art, Tsuda teaches an invention for warning a user of the occurrence of moire pattern (para 4, 7, 16-18, 26-28).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify Geissler, Rotscholl, Hirotsune, and Imoto for detecting artifacts comprising a camera to determine when artifacts occur and for mitigating interference patterns (i.e., artifacts) produced when a camera sensor captures images or video including a display screen used as a background comprising a reduction of moiré interference by smoothing filtering of a camera image wherein low-pass filters for filtering the camera image in position space are used by further incorporating known elements of Tsuda’s invention for warning a user of the occurrence of moire patterns in order to alleviate the problem associated with the produced interference patterns (artifacts/moire) during an editing process.
Claim(s) 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Geissler; Michael et al. US 20230336870 A1 and in further view of Rotscholl; Ingo et al. US 20230281758 A1 (hereafter Rotscholl) and in further view of Hirotsune; Satoshi US 20190306384 A1 (hereafter Hirotsune) and in further view of Imoto; Masayoshi et al. US 20150271409 A1 (Imoto) and in further view of Murase; Hiroshi US 20180027149 A1 (hereafter Murase).
Regarding claim 6, “wherein the determination unit performs processing of associating determination result information of the determination processing with a project” the combination of Geissler, Rotscholl, Hirotsune, and Imoto render obvious that claimed video of a camera and a display as used for generating videos in a studio against and video display backdrop and then the videos can be edited correspond to “projects” but are silent with respect to associating determination result information of the determination processing claimed. In an analogous art, Murase teaches the deficiency of Geissler, Rotscholl, Hirotsune, and Imoto (para 42-51).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify Geissler, Rotscholl, Hirotsune, and Imoto for detecting artifacts comprising a camera to determine when artifacts occur and for mitigating interference patterns (i.e., artifacts) produced when a camera sensor captures images or video including a display screen used as a background comprising a reduction of moiré interference by smoothing filtering of a camera image wherein low-pass filters for filtering the camera image in position space are used by further incorporating known elements of Murase’s invention for warning a tracking identification of artifacts in video (e.g., moire patterns) and the data necessary for correcting the artifacts in a database in order to alleviate the problem associated with the produced interference patterns (artifacts/moire) during an editing process.
Regarding claim 7, “wherein the determination unit performs processing of storing the determination result information in a storage medium in association with a project” is further rejected as discussed in the rejection of claims 1-6 wherein the combination of Geissler, Rotscholl, Hirotsune, and Imoto render obvious that claimed video of a camera and a display as used for generating videos in a studio against and video display backdrop and then the videos can be edited correspond to “projects” but are silent with respect to are silent with respect to associating determination result information of the determination processing claimed. In an analogous art, Murase teaches the deficiency of Geissler, Rotscholl, Hirotsune, and Imoto (See Murase para 42-51). See also Imoto element 30 and 53 shown as external components for storing data.
Regarding claim 8, “wherein the determination unit performs processing of transmitting the determination result information to an external device in association with a project” is further rejected as discussed in the rejection of claims 1-7 wherein the combination of Geissler, Rotscholl, Hirotsune, and Imoto render obvious that claimed video of a camera and a display as used for generating videos in a studio against and video display backdrop and then the videos can be edited correspond to “projects” but are silent with respect to are silent with respect to associating determination result information of the determination processing claimed. In an analogous art, Murase teaches the deficiency of Geissler, Rotscholl, Hirotsune, and Imoto (See Murase elements 3 and 5 and para 42-51 disclosing components for storing moire related data). See also Imoto element 30 and 53 shown as external components for storing data.
Claim(s) 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Geissler; Michael et al. US 20230336870 A1 and in further view of Rotscholl; Ingo et al. US 20230281758 A1 (hereafter Rotscholl) and in further view of Hirotsune; Satoshi US 20190306384 A1 (hereafter Hirotsune) and in further view of Imoto; Masayoshi et al. US 20150271409 A1 (Imoto) and in further view of Zink; Michael et al. US 11457187 B1 (hereafter Zink).
Regarding claim 11, “wherein the imaging system is a system that captures, with a camera, a video of a display that displays a virtual video obtained by rendering using a 3D model, and the imaging system includes a rendering unit that performs rendering using the 3D model to generate a virtual video, and a video processing unit that generates a simulation video for a virtual video generated by the rendering unit by using a processing parameter that realizes a luminance or color characteristic at a time of imaging of a camera used in the imaging system” the combination of Geissler, Rotscholl, Hirotsune, and Imoto render obvious a system that captures, with a camera, a video of a display that displays a video but are silent with respect to virtual video and 3D model as claimed. In an analogous art, Zink teaches the deficiency of Geissler, Rotscholl, Hirotsune, and Imoto (Abstract, Fig. 1a, 1b, and col. 2:61-67 to col. 6:1-18 disclosing generating a virtual video comprising IMAX 3D video; see also col. 6:28-40, col 16:1-67 to col 17:1-36, and col 17:37-67 to col. 18:1-54 disclosing luminance data for videos).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify Geissler, Rotscholl, Hirotsune, and Imoto for detecting artifacts comprising a camera to determine when artifacts occur and for mitigating interference patterns (i.e., artifacts) produced when a camera sensor captures images or video including a display screen used as a background comprising a reduction of moiré interference by smoothing filtering of a camera image wherein low-pass filters for filtering the camera image in position space are used by further incorporating known elements of Zink’s invention for an imaging system for capturing, with a camera, a video of a display that displays a virtual video obtained by rendering using a 3D model and using a processing parameter that realizes a luminance or color characteristic at a time of imaging of a camera used in the imaging system in order to alleviate the problem associated with the produced interference patterns (artifacts/moire) during an editing process.
Regarding claim 12, “wherein the imaging system is a system that captures, with a camera, a video of a display that displays a virtual video obtained by rendering using a 3D model, and the imaging system includes a rendering unit that performs rendering using the 3D model and generates a virtual video to be displayed on the display” is further rejected as discussed in the rejection of claims 1-11 wherein the combination of Geissler, Rotscholl, Hirotsune, and Imoto render obvious a system that captures, with a camera, a video of a display that displays a video but are silent with respect to virtual video and 3D model as claimed. In an analogous art, Zink teaches the deficiency of Geissler, Rotscholl, Hirotsune, and Imoto (Abstract, Fig. 1a, 1b, and col. 2:61-67 to col. 6:1-18 disclosing generating a virtual video comprising IMAX 3D video; see also col. 6:28-40, col 16:1-67 to col 17:1-36, and col 17:37-67 to col. 18:1-54 disclosing luminance data for videos).
CONCLUSION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALFONSO CASTRO whose telephone number is (571)270-3950. The examiner can normally be reached on Monday to Friday from 10am to 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nathan Flynn can be reached. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALFONSO CASTRO/Primary Examiner, Art Unit 2421
Prosecution Insights