DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed August 4th, 2026 has been entered. Claims 1-17 and 22-24 remain pending in the application. Claims 1-6, 8-13, 16 and 17 have been amended. Claims 18-21 have been canceled. Claims 22-24 have been newly added. The amendments to the Claims have overcome each and every objection previously set forth in the Non-Final Office Action mailed May 20th, 2026.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 11, 15-17 and 22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Koppetz (US 20220030139 A1).
Regarding Claim 1, Koppetz teaches An information processing apparatus (Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5)”) comprising:
Circuitry (Paragraph [0013]: “The control device can be designed, for example, as a microprocessor, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), which is configured to process the received lens data and to generate appropriate commands for adjusting the representation of the virtual background. The control device can have an integrated memory or be connected to a memory, for example, in order to be able to store and access image information and position information of the virtual background”. Notes: Circuitry is inherent to a processor)
configured to receive a first video (Paragraph [0009]: “Since the control device comprises a data input for receiving lens data of the associated camera, camera or lens-specific parameters or settings of the camera for a representation of the virtual background can be considered, so that by adjusting the representation of the virtual background, the optical image of this representation produced or producible by means of the camera can be influenced”; Paragraph [0019]: “for example while a film scene is being recorded, so that the representation can be adjusted, for example, in real time to the changing settings of the lens or camera”)
rendered using a 3D model (Paragraph [0014]: “Furthermore, it can be provided for, that the control device comprises a game-engine or is configured to use a game-engine in order to generate the representation of the virtual background. A game-engine can be realised basically as a software in order to generate a representation of an environment, in particular in a computer game taking into account a position, in particular a position of a player”; Paragraph [0018]: “In order to adjust the representation of the virtual background, it is not necessary that a complete representation of the virtual background has been generated by the background display device before the receiving of lens data or before adjusting the representation. Rather, the adjusting of the representation dependent on the received lens data can, for example, also take place at the beginning of a scene or before a recording when generating the representation, wherein the lens data can be received and the representation in dependence thereof can be generated. Thus, the adjusting of the representation can in principle also take place, or be understood as such, as an integral process during the generation of the representation by means of the background display device. To generate and adjust the representation, the control device can, for example, draw on a model, in particular a three-dimensional model, which can be stored and made available in a non-volatile memory, for example a semiconductor memory. In particular, the control device can draw on a respective model in dependence of the received information about the lens used. Alternatively, or in addition, the control device can draw on a game-engine to generate the representation”),
receive a parameter that achieves a characteristic of luminance or color at a time of imaging by a camera used in an imaging system (Paragraph [0028]: “In some embodiments, the control device can be configured to adjust the representation of the virtual background in such a way that an optical image of the representation generated by the camera in accordance with the received lens data approximates a theoretical image that would be generated if a real background corresponding to the virtual background was imaged by means of the assigned camera”; Paragraph [0030]: “For example, the sharpness of the different regions of the real background in a theoretical image, which the camera would produce when imaging the real background due to its three-dimensionality, depends on the distance between these regions and the camera and, in particular, can change dependent on the lens data or lens setting values, for example, a diaphragm aperture. In order to be able to achieve such effects in the optical image produced or producible by means of the camera, even when recording a scene before the representation of the virtual background, the representation can be adjusted, for example, so as to increase the region of blur in the optical image generated by the camera caused already by a blurred representation. In particular, such a blurring can be generated by combining a number of lighting means of an active illumination apparatus, in particular, a plurality of light emitting diodes to form an area with the same and/or similar colour and/or brightness”), and
convert, by using the parameter, the received first video into a second video having luminance or color equivalent to that of a captured video obtained by the camera capturing the first video displayed on a display (Paragraph [0031]: “For example, the control device can be configured to calculate a theoretical image, i.e. an image of a real background corresponding to a virtual background, in dependence of the received lens data, in order to be able to compare the optical image that can be generated by the camera with this lens data, with the theoretical image. For this purpose, the control device can, for example, draw on a model of the virtual background, which, in particular, can comprise position data for a plurality of (virtual) background regions, from which the distances between these background regions of the virtual background to the (real) background display device onto which the virtual background is displayed, and/or to the camera can be derived. This position data can, in particular, also take into account a curvature of the (real) background display device in a space. The control device can also be configured to calculate the actual producible image of the representation of the virtual background dependent on the lens data in order to be able to compare this image with the theoretical image, for example, using image analysis methods, and to be able to adapt the representation of the virtual background so that the optical image can approximate the theoretical image”).
Regarding Claim 2, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz teaches the information processing apparatus according to Claim 1, wherein the circuitry is further configured to perform display video processing with the parameter that achieves the characteristic of luminance or color of the video on the display (Paragraph [0011]: “The virtual background can be displayed by the background display device”; Paragraph [0014]: “The representation generated by the game-engine can then be adjusted dependent on the received lens data in order that the optical image generated by the camera approximates a theoretical image of a real background corresponding to the virtual background. Such a game-engine can run, for example, on a microprocessor (in particular, also a multi-core) or a graphics card”; Paragraph [0030]: “the sharpness of the different regions of the real background in a theoretical image, which the camera would produce when imaging the real background due to its three-dimensionality, depends on the distance between these regions and the camera and, in particular, can change dependent on the lens data or lens setting values, for example, a diaphragm aperture. In order to be able to achieve such effects in the optical image produced or producible by means of the camera, even when recording a scene before the representation of the virtual background, the representation can be adjusted, for example, so as to increase the region of blur in the optical image generated by the camera caused already by a blurred representation. In particular, such a blurring can be generated by combining a number of lighting means of an active illumination apparatus, in particular, a plurality of light emitting diodes to form an area with the same and/or similar colour and/or brightness”), and
then perform camera video processing with the parameter that achieves the characteristic of luminance or color at the time of imaging by the camera to generate the second video (Paragraph [0012]: “In that, the control device is configured to adjust the representation of the virtual background dependent on the received lens data, in particular based on the respective settings of lens parameters or other values that characterize the camera lens used, the effects arising in a theoretical image of a real background, in particular three-dimensional, corresponding to a virtual background can be reproduced in the actual generated image of the representation of the virtual background. For this purpose, the lens data can comprise in particular the lens setting values”; Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”).
Regarding Claim 11, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz teaches the information processing apparatus according to Claim 1, wherein the circuitry generates the second video reflecting lens characteristics of the camera used in the imaging system (Paragraph [0012]: “In that, the control device is configured to adjust the representation of the virtual background dependent on the received lens data, in particular based on the respective settings of lens parameters or other values that characterize the camera lens used, the effects arising in a theoretical image of a real background, in particular three-dimensional, corresponding to a virtual background can be reproduced in the actual generated image of the representation of the virtual background. For this purpose, the lens data can comprise in particular the lens setting values”; Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”).
Regarding Claim 15, the information processing apparatus according to Claim 1 is rejected over Koppetz as modified.
Kopetz as modified teaches the information processing apparatus according to Claim 1, wherein the information processing apparatus is a device separate from a device constituting the imaging system (Paragraph [0016]: “the objects in the optical image of the representation of the virtual background generated by the camera are imaged mostly having a substantially equal sharpness determined by the distance of the background display device to the camera”; Paragraph [0010]: “To represent the virtual background, the background display device can constitute, for example, an electronic display with an active pixel matrix and, in particular, can comprise an active illumination apparatus having a plurality of light sources, in particular an LED wall with a plurality of light-emitting diodes or an OLED wall with a plurality of light-emitting diodes which can be controlled individually and/or in groups of adjacent lighting means or arrays of lighting means”. Notes: The broadest reasonable interpretation of a device separate from another device is that the devices are physically separated from one another. Hence, the camera, which is considered the imaging system, is separate from the LED background wall (display) which renders the virtual video, and is hence considered physically separate from the information processing apparatus).
Claim 16, being similar in scope to Claim 1, is rejected under the same rationale.
Claim 17, being similar in scope to Claim 1, is rejected under the same rationale.
Regarding Claim 22, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz teaches the information processing apparatus according to Claim 1, wherein the circuitry is further configured to generate the first video by performing rendering using the 3D model (Paragraph [0014]: “Furthermore, it can be provided for, that the control device comprises a game-engine or is configured to use a game-engine in order to generate the representation of the virtual background. A game-engine can be realised basically as a software in order to generate a representation of an environment, in particular in a computer game taking into account a position, in particular a position of a player”; Paragraph [0018]: “In order to adjust the representation of the virtual background, it is not necessary that a complete representation of the virtual background has been generated by the background display device before the receiving of lens data or before adjusting the representation. Rather, the adjusting of the representation dependent on the received lens data can, for example, also take place at the beginning of a scene or before a recording when generating the representation, wherein the lens data can be received and the representation in dependence thereof can be generated. Thus, the adjusting of the representation can in principle also take place, or be understood as such, as an integral process during the generation of the representation by means of the background display device. To generate and adjust the representation, the control device can, for example, draw on a model, in particular a three-dimensional model, which can be stored and made available in a non-volatile memory, for example a semiconductor memory. In particular, the control device can draw on a respective model in dependence of the received information about the lens used. Alternatively, or in addition, the control device can draw on a game-engine to generate the representation”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3-5, 10, and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Koppetz (US 20220030139 A1) in view of Zhao (CN114051129A).
Regarding Claim 3, Koppetz teaches the information processing apparatus according to Claim 1.
Koppetz teaches color adjustment processing for a second video according to a control signal (Paragraph [0012]: “In that, the control device is configured to adjust the representation of the virtual background dependent on the received lens data, in particular based on the respective settings of lens parameters or other values that characterize the camera lens used, the effects arising in a theoretical image of a real background, in particular three-dimensional, corresponding to a virtual background can be reproduced in the actual generated image of the representation of the virtual background. For this purpose, the lens data can comprise in particular the lens setting values”; Paragraph [0030]: “the sharpness of the different regions of the real background in a theoretical image, which the camera would produce when imaging the real background due to its three-dimensionality, depends on the distance between these regions and the camera and, in particular, can change dependent on the lens data or lens setting values, for example, a diaphragm aperture. In order to be able to achieve such effects in the optical image produced or producible by means of the camera, even when recording a scene before the representation of the virtual background, the representation can be adjusted, for example, so as to increase the region of blur in the optical image generated by the camera caused already by a blurred representation. In particular, such a blurring can be generated by combining a number of lighting means of an active illumination apparatus, in particular, a plurality of light emitting diodes to form an area with the same and/or similar colour and/or brightness”; Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”), and
generate color conversion information based on the color adjustment processing (Paragraph [0112]: “the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”)
Koppetz does not explicitly teach that color adjustment processing for the second video is according to a control signal based on user operation.
However, Zhao teaches color adjustment processing for a video is according to a control signal based on user operation (Summary of the Invention, pg 3: “During the shooting process, the virtualization manufacturing team <read on user> adjusting the real-time rendering engine module through the background control module <read on control signal> according to the requirements of the main team, including a change of the virtual scene content, a change of brightness and color of the digital light device, a change of brightness and color of the digital light array, and a change of brightness, color, and pattern of the LED screen”; Summary of the Invention, pg 3: “According to the movie virtualization manufacturing system and method based on the LED background wall of the present invention, the digital light array is used to replace the top surface LED background wall as an illumination scheme, and this illumination scheme has stronger color rendering than the illumination scheme of the top surface LED background wall, provides a higher shooting brightness for the shooting scene, and can control the illumination of the scene more flexibly according to the requirements of the driver”; Detailed Description of the Embodiments, pg 5: “In an embodiment, the video processing distribution system includes: a 4K video processor…The video processing and distribution system obtains a rendered picture of each render child nodethrough the HDMI video line, and outputs the rendered picture to the LED screen module through the network line”. Notes: The user adjusts color and brightness through the background control module, which necessarily requires a control signal for operation; the adjusted color and brightness are subsequently processed through the processing parameters of the camera that captures the simulation video).
Koppetz and Zhao are considered analogous in the art with respect to color adjustment processing for videos according to a control signal. A common motivation in the art is to have a higher degree of human control over processes involving visual balance, which is commonly achieved through user input in systems, as is evident in Zhao.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the information processing apparatus with color adjustment processing of Koppetz with a control signal based on user operation for color adjustment processing of Zhao; Doing so would yield the predictable result of enabling a higher degree of user control on the color adjustment processing of Koppetz.
Regarding Claim 4, the information processing apparatus according to Claim 3 is rejected over Koppetz as modified.
Koppetz as modified teaches the information processing apparatus according to Claim 3, wherein the circuitry is further configured to control output of the color conversion information (Koppetz, Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”. Notes: The circuitry is inherently configured to control output of the color conversion information because it uses the color conversion information to adjust the virtual background (video)).
Regarding Claim 5, the information processing apparatus according to Claim 4 is rejected over Koppetz as modified.
Koppetz as modified teaches the information processing apparatus according to Claim 4, wherein the color conversion information is output as information used for signal processing for the captured video obtained by the camera (Koppetz, Paragraph [0028]: “In some embodiments, the control device can be configured to adjust the representation of the virtual background in such a way that an optical image of the representation generated by the camera in accordance with the received lens data approximates a theoretical image that would be generated if a real background corresponding to the virtual background was imaged by means of the assigned camera”; Koppetz, Paragraph [0030]: “For example, the sharpness of the different regions of the real background in a theoretical image, which the camera would produce when imaging the real background due to its three-dimensionality, depends on the distance between these regions and the camera and, in particular, can change dependent on the lens data or lens setting values, for example, a diaphragm aperture. In order to be able to achieve such effects in the optical image produced or producible by means of the camera, even when recording a scene before the representation of the virtual background, the representation can be adjusted, for example, so as to increase the region of blur in the optical image generated by the camera caused already by a blurred representation. In particular, such a blurring can be generated by combining a number of lighting means of an active illumination apparatus, in particular, a plurality of light emitting diodes to form an area with the same and/or similar colour and/or brightness”; Koppetz, Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”).
Regarding Claim 10, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz does not explicitly teach that the circuitry generates the second video with an angle of view setting based on angle of view information at the time of imaging by the camera used in the imaging system.
However, Zhao teaches generating video with an angle of view setting based on angle of view information at the time of imaging by the camera used in the imaging system (Summary of the Invention, pg 2: “the position and posture <read on angle of view> of the camera are simulated in real time by controlling the position and posture of the virtual camera”; Detailed Description of the Embodiments, pg 5: “In an embodiment, the video processing distribution system includes: a 4K video processor…The video processing and distribution system obtains a rendered picture of each render child node through the HDMI video line, and outputs the rendered picture to the LED screen module through the network line”).
Koppetz and Zhao are considered analogous in the art with respect to image processing for videos according to a control signal. A common motivation in the art is to maximize continuity between corresponding video/image captures, especially with regards to the generation of a video have a simulation video match the orientation of an initial video, as is evident in Zhao.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the information processing apparatus of Koppetz with the video generation with an angle of view setting based on angle of view information at imaging of Zhao; Doing so would yield the predictable result of a second video based on a first video matching with regards to an angle of view.
Regarding Claim 23, the information processing method according to Claim 16 is rejected over Koppetz.
Koppetz teaches the information processing method according to Claim 16, further comprising:
Performing color adjustment processing for the second video according to a control signal (Paragraph [0012]: “In that, the control device is configured to adjust the representation of the virtual background dependent on the received lens data, in particular based on the respective settings of lens parameters or other values that characterize the camera lens used, the effects arising in a theoretical image of a real background, in particular three-dimensional, corresponding to a virtual background can be reproduced in the actual generated image of the representation of the virtual background. For this purpose, the lens data can comprise in particular the lens setting values”; Paragraph [0030]: “the sharpness of the different regions of the real background in a theoretical image, which the camera would produce when imaging the real background due to its three-dimensionality, depends on the distance between these regions and the camera and, in particular, can change dependent on the lens data or lens setting values, for example, a diaphragm aperture. In order to be able to achieve such effects in the optical image produced or producible by means of the camera, even when recording a scene before the representation of the virtual background, the representation can be adjusted, for example, so as to increase the region of blur in the optical image generated by the camera caused already by a blurred representation. In particular, such a blurring can be generated by combining a number of lighting means of an active illumination apparatus, in particular, a plurality of light emitting diodes to form an area with the same and/or similar colour and/or brightness”; Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”);
Generating color conversion information based on the color adjustment processing (Paragraph [0112]: “the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”); and
Outputting the color conversion information as information used for signal processing for a captured video by the camera (Koppetz, Paragraph [0028]: “In some embodiments, the control device can be configured to adjust the representation of the virtual background in such a way that an optical image of the representation generated by the camera in accordance with the received lens data approximates a theoretical image that would be generated if a real background corresponding to the virtual background was imaged by means of the assigned camera”; Koppetz, Paragraph [0030]: “For example, the sharpness of the different regions of the real background in a theoretical image, which the camera would produce when imaging the real background due to its three-dimensionality, depends on the distance between these regions and the camera and, in particular, can change dependent on the lens data or lens setting values, for example, a diaphragm aperture. In order to be able to achieve such effects in the optical image produced or producible by means of the camera, even when recording a scene before the representation of the virtual background, the representation can be adjusted, for example, so as to increase the region of blur in the optical image generated by the camera caused already by a blurred representation. In particular, such a blurring can be generated by combining a number of lighting means of an active illumination apparatus, in particular, a plurality of light emitting diodes to form an area with the same and/or similar colour and/or brightness”; Koppetz, Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”)
Koppetz does not explicitly teach that color adjustment processing for the second video is according to a control signal based on user operation.
However, Zhao teaches color adjustment processing for a video is according to a control signal based on user operation (Summary of the Invention, pg 3: “During the shooting process, the virtualization manufacturing team <read on user> adjusting the real-time rendering engine module through the background control module <read on control signal> according to the requirements of the main team, including a change of the virtual scene content, a change of brightness and color of the digital light device, a change of brightness and color of the digital light array, and a change of brightness, color, and pattern of the LED screen”; Summary of the Invention, pg 3: “According to the movie virtualization manufacturing system and method based on the LED background wall of the present invention, the digital light array is used to replace the top surface LED background wall as an illumination scheme, and this illumination scheme has stronger color rendering than the illumination scheme of the top surface LED background wall, provides a higher shooting brightness for the shooting scene, and can control the illumination of the scene more flexibly according to the requirements of the driver”; Detailed Description of the Embodiments, pg 5: “In an embodiment, the video processing distribution system includes: a 4K video processor…The video processing and distribution system obtains a rendered picture of each render child nodethrough the HDMI video line, and outputs the rendered picture to the LED screen module through the network line”. Notes: The user adjusts color and brightness through the background control module, which necessarily requires a control signal for operation; the adjusted color and brightness are subsequently processed through the processing parameters of the camera that captures the simulation video).
Koppetz and Zhao are considered analogous in the art with respect to color adjustment processing for videos according to a control signal. A common motivation in the art is to have a higher degree of human control over processes involving visual balance, which is commonly achieved through user input in systems, as is evident in Zhao.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the information processing apparatus with color adjustment processing of Koppetz with a control signal based on user operation for color adjustment processing of Zhao; Doing so would yield the predictable result of enabling a higher degree of user control on the color adjustment processing of Koppetz.
Claim 24, being similar in scope to Claim 23, is rejected under the same rationale
Claims 6-7 is rejected under 35 U.S.C. 103 as being unpatentable over Koppetz (US 20220030139 A1) in view of Ho (US 10126913 B1).
Regarding Claim 6, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz teaches the information processing apparatus according to Claim 1, wherein the circuitry is further configured to perform processing of changing camera setting information to be reflected in the second video according to a control signal (Koppetz, Paragraph [0030]: “For example, the sharpness of the different regions of the real background in a theoretical image, which the camera would produce when imaging the real background due to its three-dimensionality, depends on the distance between these regions and the camera and, in particular, can change dependent on the lens data or lens setting values, for example, a diaphragm aperture. In order to be able to achieve such effects in the optical image produced or producible by means of the camera, even when recording a scene before the representation of the virtual background, the representation can be adjusted, for example, so as to increase the region of blur in the optical image generated by the camera caused already by a blurred representation. In particular, such a blurring can be generated by combining a number of lighting means of an active illumination apparatus, in particular, a plurality of light emitting diodes to form an area with the same and/or similar colour and/or brightness”; Koppetz, Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”), and
control output of the changed camera setting information (Koppetz, Paragraph [0112]: “To address this problem, as shown in FIG. 4, the background display system 11 comprises a control device 25 which is connected to the background display device 15 and is configured to control the background display device 15. Thereby, the control device 25 which can, for example, be designed as a microprocessor, comprises a data input 27 in order to receive lens data 29 transmitted by the camera 23, and in addition, the control device 25 is configured to adjust the representation 19 of the virtual background 21 dependent on the received lens data 29 (cf. also FIG. 5).”. Notes: The circuitry is inherently configured to control output of the changed camera setting information because it uses the changed camera setting information to adjust the virtual background (video)).
Koppetz does not teach that the changing camera setting information according to a control signal is based on user operation.
However, Ho teaches that changing camera setting information according to a control signal is based on a user operation (Column 8, lines 12-14: “For example, a user may interact with a designated area of the screen or use a keyboard to change the camera's position or orientation”. Notes: a control signal is inherent to a user changing a camera’s position via a user interface on a screen).
Koppetz and Ho are considered analogous in the art with respect to changing camera setting information according to a control signal. A common motivation in the art is to have a higher degree of human control over processes involving camera settings that can impact the image output, which is commonly achieved through user input in systems, as is evident in Ho.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing apparatus that changes camera setting information according to a control signal of Koppetz with the changing of camera setting information according to a control signal based on user input of Ho; Doing so would yield the predictable result of enabling a higher degree of user control on camera settings.
Regarding Claim 7, the information processing apparatus according to Claim 6 is rejected over Koppetz as modified.
Koppetz as modified teaches the information processing apparatus according to Claim 6, wherein the camera setting information is output as information used for setting of the camera (Ho, Column 8, lines 12-14: “For example, a user may interact with a designated area of the screen or use a keyboard to change the camera's position or orientation”. Notes: the camera setting information is necessarily output as information used for setting of the camera considering the user can change the camera’s position or orientation through a control signal based on user input).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Koppetz (US 20220030139 A1) in view of Nakada (US 20210127103 A1).
Regarding Claim 8, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz does not teach that a white balance value is included as the parameter that achieves the characteristic of luminance or color at the time of imaging by the camera used in the imaging system.
However, Nakada teaches a white balance value is included as the parameter that achieves the characteristic of luminance or color at the time of imaging by the camera used in the imaging system (Paragraph [0037]: “In step S402, the development parameter setting unit 301 sets the development parameters <read on processing parameter> based on the imaging information contained in the RAW image data acquired from the RAM 203 and the user operation acquired from the control unit 205. FIG. 6 illustrates an example of a screen for setting the white balance included in the development parameters according to the present exemplary embodiment. In the present exemplary embodiment, white balance values are stored in advance in the ROM 202 or the like; the white balance values correspond to various types of light sources, such as sunlight, an incandescent lamp, and a fluorescent lamp. The development parameter setting unit 301 acquires a white balance value corresponding to the light source selected based on the user operation among the various types of light sources displayed on the white balance setting screen”).
Koppetz and Nakada are considered analogous in the art with respect to color adjustment processing of an image. It is well known in the art that white balancing is often performed on acquired image data during processing in accordance with a light source or light condition utilized, as is evident in Nakada.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the color adjustment processing of Koppetz with the use of a white balance value as a processing parameter during color adjustment processing of Nakada; Doing so would yield the predictable result of an output video that has been properly adjusted in accordance with a light source or condition.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Koppetz (US 20220030139 A1) in view of Watanabe (US 20150097977 A1).
Regarding Claim 9, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz does not explicitly teach that a value of a gain for a pixel signal of each of a plurality of colors read from an image sensor in the camera is included as the processing parameter that achieves the characteristic of luminance or color at the time of imaging by the camera used in the imaging system.
However, Watanabe teaches that a value of a gain for a pixel signal of each of a plurality of colors read from an image sensor in the camera is included as the processing parameter that achieves the characteristic of luminance or color at the time of imaging by the camera used in an imaging system (Paragraph [0031]: “The image pickup element 113 includes a plurality of pixels, and photoelectrically converts the optical image formed by the image pickup optical system 101 and then supplies a pixel signal (a captured image) acquired from the plurality of pixels to a signal processing unit 114. The signal processing unit 114 performs various known signal processing such as gain control, color balance control, and .gamma. (gamma) correction for the pixel signal output from the image pickup element 113 to convert the pixel signal into a video signal”. Notes: achieving a characteristic of luminance or color of capture is inherent to processing when converting pixel signals to a video signal, or in other words, accurately depicting a captured scene for display).
Koppetz and Watanabe are considered analogous in the art with respect to the processing of image data captured by an image sensor (camera). It is well known in the art to process captured image data for display as a video signal, of which the value of a gain for a pixel signal of the image data is a well-known parameter when converting to a video signal, as is evident in Watanabe.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the information processing apparatus of Koppetz with the use of gain values for pixel signals of colors read from an image sensor as a processing parameter for achieving luminance or color at the time of imaging of Watanabe.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Koppetz (US 20220030139 A1) in view of Komiyama (US 20210356721 A1).
Regarding Claim 12, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Koppetz does not teach that the circuitry generates the second video reflecting characteristics of an optical filter mounted on the camera used in the imaging system.
However, Komiyama teaches video reflecting characteristics of an optical filter mounted on the camera used in the imaging system (Paragraph [0032]: “SP represents a diaphragm (aperture stop). IP represents an image plane, on which an imaging plane of an image sensor (photoelectric conversion element), such as a CCD sensor or a CMOS sensor, is disposed when the zoom lenses 1a to 1i according to the examples are used as the imaging optical system for a digital still camera or a digital video camera. When the zoom lenses 1a to 1i according to the examples are used as the imaging optical system for the film-based camera, a photosensitive surface corresponding to the film surface is placed on the image plane IP. Arrows shown in each sectional view indicate moving directions of lens units during zooming from the wide-angle end to the telephoto end. FL represents an optical block corresponding to an optical filter, a face plate, a crystal low-pass filter, an infrared cut filter, and the like”; Figure 1 illustrates the aforementioned components. Notes: the video/image inherently reflects characteristics of a filter, as the use of a filter is for altering the appearance of video/image).
Koppetz and Komiyama are considered analogous in the art with respect to the use of video cameras for capturing a scene. One would be motivated to utilize an optical filter mounted on the camera, such as over the lens of the camera, to change the appearance of the scene when the image is captured, such as in general color or hue via a tinted lens; A benefit for doing so would be to reduce the harshness of light in a scene, or otherwise adapt the colors and hue of a scene for a thematic purpose.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing apparatus with an attached camera/imaging system of Koppetz with the use of an optical filter mounted on the camera of Komiyama; Doing so would yield the predictable result of generating a simulated video with the visual effects provided by a mounted optical filter.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Koppetz (US 20220030139 A1) in view of Kubota (JP2010010881A) and Tsukioka (US 20030231251 A1).
Regarding Claim 13, the information processing apparatus according to Claim 1 is rejected over Koppetz.
Zhao does not teach that moire warning output processing is performed for the first video.
However, Kubota teaches that moire warning output processing is performed for a video rendered by a rendering unit in a live view image (Paragraph [0064]: “In S408, the CPU 146 determines whether or not moire is generated in the live view image. If it is determined in S408 that no moire is generated in the live view image, the process proceeds to S610. When the moire warning display is displayed on the display element 128 in the previous processing, the moire warning display is erased”).
Koppetz and Kubota are considered analogous in the art with respect to real time adjustment to video. It is well known in the art that a moire effect may occur in a generated live view image, as is evident in Kubota. One would be motivated to check whether a moire effect is occurring to prevent diminished video output quality.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the information processing apparatus with live video processing of Koppetz with the moire check and warning output of Kubota; Doing so would yield the predictable result of enabling a live check for a moire effect in a captured video.
Koppetz as modified does not explicitly teach that moire warning output processing performed according to moire prediction information.
However, Tsukioka teaches moire prediction information (Paragraph [0051]: “In particular, unlike the first embodiment as shown in FIG. 6, a moire prediction circuit 212 and filtering circuit 213 are newly added to the interior of an intermittence interpolation circuit 205. A major difference in construction is that the signals from the moire prediction circuit 212 are fed back to the drive control circuit 204 so as to change the thinning out pattern according to the outcome of moire prediction”).
Koppetz as modified and Tsukioka are considered analogous in the art with respect to processing moire in images/videos. One would be motivated to combine moire prediction information with the output of a moire warning, as doing so would enable a viewer/user of the apparatus to be warned during live video or before the appearance of a moire effect; in doing so, adjustments to video processing may be made accordingly to prevent diminishing video quality.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the moire warning output processing of Zhao as modified with the moire prediction of Tsukioka; Doing so would yield the predictable result of warning a user or viewer of an image/video that a moire effect may occur.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Koppetz (US 20220030139 A1) in view of Kubota (JP2010010881A) and Tsukioka (US 20030231251 A1), and in further view of Ho (US 10126913 B1).
Regarding Claim 14, the information processing apparatus according to Claim 13 is rejected over Koppetz as modified.
Koppetz as modified teaches that moire prediction is performed according to the camera position that can be changed arbitrarily on a video (Tsukioka, Paragraph [0051]: “In particular, unlike the first embodiment as shown in FIG. 6, a moire prediction circuit 212 and filtering circuit 213 are newly added to the interior of an intermittence interpolation circuit 205. A major difference in construction is that the signals from the moire prediction circuit 212 are fed back to the drive control circuit 204 so as to change the thinning out pattern according to the outcome of moire prediction”; Kubota, Paragraph [0064]: “In S408, the CPU 146 determines whether or not moire is generated in the live view image. If it is determined in S408 that no moire is generated in the live view image, the process proceeds to S610. When the moire warning display is displayed on the display element 128 in the previous processing, the moire warning display is erased”; Koppetz, Paragraph [0037]: “Also, by taking into account such position data or orientations of the camera when adjusting the representation, an optical image of the representation of the virtual background generated by means of the camera can approximate a theoretical image that would result from a recording of a real background corresponding to the virtual background. For example, when the camera is pivoted, angles to objects displayed by means of the background display device can change differently than would be the case with objects in a real background in which the objects show different distances to the camera”. Notes: moire prediction is performed according to an acquired image/video, which inherently reflects the camera position from which the image/video was taken; hence, moire prediction is performed according to the camera position, which as demonstrated by Zhao, can be changed accordingly).
Koppetz as modified does not teach a user interface prepared for arbitrarily changing a camera position.
However, Ho teaches a user interface prepared for arbitrarily changing a camera position (Column 8, lines 12-14: “For example, a user may interact with a designated area of the screen or use a keyboard to change the camera's position or orientation”. Notes: a user interface is inherent to a user interacting with the screen to change the camera’s position or orientation).
Koppetz as modified and Ho are considered analogous in the art with respect to the changing of a camera’s position or orientation. A common motivation for utilizing a user interface for performing a task is to make performing the task more intuitive or simple for a user. Hence, a person having ordinary skill in the art would be motivated to utilize a user interface as a mechanism to change a camera’s position to simplify and increase the ease of performing the task of changing a camera’s position.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the changing of a camera position of Koppetz as modified with the use of a user interface to change camera position of Ho; Doing so would yield the predictable result of improving ease of use of the apparatus when moving a camera position.
Response to Arguments
Applicant’s arguments, see Applicant Remarks, pg. 13-15, filed August 4th, 2026, with respect to the rejections of Claims 1, 16-17 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection, necessitated by amendment, is made in view of Koppetz (US 20220030139 A1). For more detail on how Koppetz teaches amended Claims 1 and 15-17, refer to the Rejection of Claims 1, from which 16-17 are similarly rejected under similar rationale.
All dependent Claims of Independent Claims 1 and 16-17 are not considered patentable solely based on the patentability of Independent Claims 1 and 16-17. In view of the new grounds of rejection necessitated by amendment of Claims 1 and 16-17, all arguments regarding dependent Claims 2-14 and 15 of Independent Claims 1 and 16-17 are similarly considered persuasive, owing to the arguments for Claims 2-14 and 15 being centered on the amended limitations introduced in Independent Claims 1, 16-17, and withdrawn. New grounds of rejection are formed for all dependent Claims 2-14 and 15 necessitated by amendment. Refer to the Rejections of Claims 2-14 and 15 for more detail.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614