DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is in response to Applicant’s amendment/response filed on 21 May 2026, which has been entered and made of record.
Double Patenting
The previous nonstatutory double patenting rejection is withdrawn in view of the approved Terminal Disclaimer.
Response to Arguments
Applicant's arguments filed 21 May 2026 have been fully considered but they are not persuasive.
Applicant argues “In Iwase … this audio generation and control signal embedding all occur within the same guitar device. There is no separate ‘sound collecting device’ and ‘lighting control device’ as recited in claims 1 and 15” (Remarks, pg. 8). The Examiner responds that this argument does not correspond to the manner in which the references are used in the rejection, i.e. Islam modified by Iwase. The primary reference Islam generates audio and control/lighting data from separate devices, which is not in dispute, and transmits those signals to a remote environment for reproduction of the sound and lighting. Iwase teaches that, when transmitting sound and control/lighting signals, it is obvious to embed the control/lighting signals into the sound signal for efficiency. When that signal embedding of Iwase is applied to Islam, the combination would render obvious the original system of Islam where the audio and control/lighting signal generation occurs on separate devices, but the signals would be transmitted in a superimposed manner as taught by Iwase.
Applicant argues “in Iwase, the embedding occurs during audio generation, not ‘after the sound signal is acquired’ as recited in claims 1 and 15 … The superimposing section receives the audio signal directly from the musical sound generating section and immediately superimposes the control data” (Remarks, pg. 8). The Examiner respectfully disagrees. Iwase recites “superimpose the musical performance information [i.e. the control/lighting data] on the audio signal” (para. 98). In order to superimpose control/lighting data onto a sound signal, the sound signal must necessarily already be present. Therefore, the embedding occurs after the sound signal is acquired.
Applicant argues “Islam fails to disclose embedding a lighting signal into a sound signal, let alone embedding by a lighting control device separate from a sound collecting device after the sound signal is acquired” (Remarks, pg. 9). This argument is moot because Iwase is used in the rejection to teach embedding a lighting signal onto a sound signal, and when that is applied to the separately-generated sound and lighting signals of Islam, the combination of references renders obvious the disputed claim limitations.
Applicant argues “Neither Islam nor Iwase teaches this extraction step. In Iwase, the control data is used internally within the instrument and there is no extraction of an embedded lighting signal from a collected sound signal” (Remarks, pg. 9). The Examiner respectfully disagrees. Iwase teaches outputting the superimposed signals through an output device and subsequently inputting those signals in a reproducing device (see Fig. 3). “The decoding section [of the reproducing device] extracts and decodes the musical performance information [i.e. the control/lighting signal] superimposed on the audio signal” (Iwase, para. 77). This teaches the limitations of new claim 16.
Applicant argues “Neither Islam nor Iwase teaches customizing the embedded lighting signal based on the user's lighting device environment” (Remarks, pg. 9). The Examiner respectfully disagrees. As a first matter, the claim limitation does not recite “customizing,” but rather that the signal is embedded “according to” an environment of the second lighting device, which is broader than customizing. Islam as modified by Iwase teaches generating sound and lighting signals at a first environment, embedding the lighting signal onto the sound signal, and transmitting the embedded signal to a second, remote environment to reproduce the sound and lighting of the first environment. In any system that receives and utilizes such signals (whether they are separate sound and lighting signals as in the unmodified Islam, or a combined sound and lighting signal as taught by Iwase and used to modify the Islam reference), the signals would have to be compatible with the reproducing device located at the second, remote environment. Therefore, the lighting signal would be embedded into the sound signal “according to an environment of the second lighting device.”
Any remaining arguments are considered moot based on the foregoing.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-8, 10-12, and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Islam et al. (US 2004/0032536; hereinafter “Islam”) in view of Iwase et al. (US 2011/0023691; hereinafter “Iwase”).
Regarding claim 1, Islam teaches A computer-implemented light emission control method (“controls lighting,” para. 7) comprising: acquiring a sound signal indicating sound collected in a target space by a sound collecting device (“high definition audio signal recorded at the live stage,” para. 46), wherein a lighting signal generated by a lighting control device separate from the sound collecting device that collects the sound signal, the lighting signal being a signal for controlling a first lighting device provided in the target space (“a lighting board at the live stage controls lighting trusses at the live stage,” para. 7); and controlling light to be emitted by a second lighting device based on the lighting signal … the second lighting device being provided in a remote location that is remote from the target space (“a lighting board at the live stage controls lighting trusses at the live stage as well as the virtual stages, in order to duplicate the lighting effects of the live performance at the virtual stages,” para. 7).
Islam does not disclose wherein the lighting signal is embedded in the sound signal after the sound signal is acquired.
In the same art of transmitting live performance data, Iwase teaches wherein a lighting signal is embedded in the sound signal after the sound signal is acquired (“The control device (musical performance-related information output device) receives a manipulation input for controlling an external apparatus (for example, … a stage-related device, such as an illumination or a camera, or the like). The control device generates a control signal, which controls the external apparatus, in accordance with the manipulation input. Then, the control device superimposes the control signal on the audio signal,” para. 199; in order to superimpose a control/lighting signal onto a sound signal, the sound signal must already be acquired, otherwise the control/lighting signal would need to be superimposed onto something that doesn’t exist).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Iwase to Islam. The motivation would have been “The control device can easily control an external apparatus connected thereto only by outputting the audio signal on which the control signal is superimposed” (Iwase, para. 200).
Regarding claim 2, the combination of Islam and Iwase renders obvious acquiring state information indicating a state of the target space; and controlling the light to be emitted by the second lighting device based on the state information (“a lighting board at the live stage controls lighting trusses at the live stage as well as the virtual stages, in order to duplicate the lighting effects of the live performance at the virtual stages,” Islam, para. 7).
Regarding claim 3, the combination of Islam and Iwase renders obvious acquiring image information indicating a target image in which the target space is displayed; and displaying a virtual space on a display device provided in the remote location based on the image information, the virtual space in which the target image is displayed (“a high definition camera and a high definition audio recording system at the live stage. Signals from the camera and audio recording system may be reproduced on high definition projectors and surround sound audio systems which are specifically configured at the virtual stages,” Islam, para. 7).
Regarding claim 6, the combination of Islam and Iwase renders obvious wherein the state information comprises information indicating a state of sound or light in the target space (“the lighting effects of the live performance,” Islam, para. 7).
Regarding claim 7, the combination of Islam and Iwase renders obvious wherein the state information is generated based on the lighting signal for controlling the first lighting device provided in the target space (“a lighting board at the live stage controls lighting trusses at the live stage as well as the virtual stages, in order to duplicate the lighting effects of the live performance at the virtual stages,” Islam, para. 7).
Regarding claim 8, the combination of Islam and Iwase renders obvious wherein the state information further comprises information for indicating a representative color representing a color of a target image in which the target space is displayed (“a high definition camera … Signals from the camera and audio recording system may be reproduced on high definition projectors,” para. 7; an image from a high definition camera would include color data).
Regarding claim 10, the combination of Islam and Iwase renders obvious controlling sound to be output from a speaker device provided in the remote location based on the sound signal (“Signals from the camera and audio recording system may be reproduced on high definition projectors and surround sound audio systems which are specifically configured at the virtual stages,” Islam, para. 7).
Regarding claim 11, the combination of Islam and Iwase renders obvious wherein the first lighting device is a physical lighting device (“a lighting board at the live stage controls lighting trusses at the live stage,” Islam, para. 7).
Regarding claim 12, the combination of Islam and Iwase renders obvious wherein the second lighting device is a physical lighting device (“the virtual stage 102 [of Fig. 3] … include a trough 308 between the stage 102 and the screen 304 to minimize any spill light from a downstage lighting truss 310 onto the screen 304,” Islam, para. 33; this description shows that the lighting truss 310 of Fig. 3 at the virtual stage is a physical lighting device because it would not be necessary to “minimize any spill light” if the light was a virtual light).
Regarding claim 14, the combination of Islam and Iwase renders obvious wherein the lighting signal is separate from the target image (“generate a high definition audio signal,” Islam, para. 28; “project a high definition [image] signal with sufficient resolution to create an illusion of a live performance at a distance of 10 feet or greater from the projection screen,” Islam, para. 35; these two recitations from Islam show that the audio signal is separate from the image signal, and therefore in the combination of Islam and Iwase, where the lighting signal is embedded in the audio signal, the lighting signal would also be separate from the image signal).
Regarding claim 15, it is rejected using the same citations and rationales described in the rejection of claim 1, with the additional limitations of A control system comprising: a processor; and a memory storing instructions that, when executed by the processor, cause the control system to perform operations (“lighting boards at the live and virtual stages are controlled by identical commands,” Islam, para. 48; processor/memory are implicit in the electronic control devices of Islam and Iwase).
Regarding claim 16, the combination of Islam and Iwase renders obvious extracting the lighting signal … and generating a lighting control signal for the second lighting device based on the extracted lighting signal (“a lighting board at the live stage controls lighting trusses at the live stage as well as the virtual stages, in order to duplicate the lighting effects of the live performance at the virtual stages,” Islam, para. 7), wherein the lighting signal is extracted from the sound signal (“The decoding section extracts and decodes the musical performance information [i.e. the control/lighting signal] superimposed on the audio signal,” Iwase, para. 77; see claim 1 for motivation to combine).
Regarding claim 17, the combination of Islam and Iwase renders obvious wherein the lighting signal is embedded in the sound signal according to an environment of the second lighting device (“Although a musical piece based on the recorded audio signal can be reproduced by using a general reproducing device, here, a method will be described which reproduces the recorded audio signal by using a reproducing device 3 capable of decoding the musical performance information superimposed on the audio signal,” Iwase, para. 72; see claim 1 for motivation to combine; the superimposed lighting/sound signal of Iwase would have to be compatible with the reproducing device, which is part of the environment of the second lighting device, and therefore the embedding happens “according to an environment of the second lighting device”).
Claims 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Islam and Iwase, and further in view of Varshney et al. (US 2020/0118342; hereinafter “Varshney”).
Regarding claim 4, the combination of Islam and Iwase does not disclose acquiring position information indicating a viewing position in the virtual space; controlling the light to be emitted by the second lighting device based on the position information.
In the same art of transmitting live performance data, Varshney teaches acquiring position information indicating a viewing position in the virtual space (“the users' current viewpoint,” para. 124); controlling the light to be emitted by the second lighting device based on the position information (“use the 360 image/video at the users' current viewpoint as a lighting source when lighting virtual objects,” para. 124).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Varshney to the combination of Islam and Iwase. The motivation would have been “to create a photorealistic real environment which is dynamic and also navigable by a user” (Varshney, para. 3).
Regarding claim 5, the combination of Islam and Iwase does not disclose acquiring position information indicating a viewing position in the virtual space; displaying the virtual space on the display device based on the position information.
In the same art of transmitting live performance data, Varshney teaches acquiring position information indicating a viewing position in the virtual space (“the users' current viewpoint,” para. 124); displaying the virtual space on the display device based on the position information (“During rendering, the position of the virtual camera which may be used to render virtual objects is set to the estimated 360 camera position of current 360 viewpoint of the user,” para. 123).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Varshney to the combination of Islam and Iwase. The motivation would have been “to create a photorealistic real environment which is dynamic and also navigable by a user” (Varshney, para. 3).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Islam and Iwase, and further in view of Charlton et al. (US 2021/0074069; hereinafter “Charlton”).
Regarding claim 9, the combination of Islam and Iwase does not disclose wherein the state information further comprises information for indicating a color corresponding to a frequency of the sound collected in the target space.
In the same art of telecommunication and mixed reality, Charlton teaches wherein the state information further comprises information for indicating a color corresponding to a frequency of the sound collected in the target space (“analyzes the audio data to determine changes in at least one of the one or more audio characteristics of the audio data. For example, the analysis component may identify a change in volume detected in the audio data such as a transition within a song during a concert ... changes in frequency," para. 55; "a change in volume or frequency of the audio data may cause the components of the augmented reality system to modify the first graphical element (e.g., the aurora) to change opacity value, color value," para. 66).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Charlton to the combination of Islam and Iwase. The motivation would have been “to improve video communications between devices” (Charlton, para. 4).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Islam and Iwase, and further in view of Mcnelley et al. (US 2019/0227419; hereinafter “Mcnelley”).
Regarding claim 13, the combination of Islam and Iwase does not disclose wherein the second lighting device is a virtual lighting device provided in the virtual space.
In the same art of simulating a live stage performance, Mcnelley teaches wherein the second lighting device is a virtual lighting device provided in the virtual space (“Virtual stage lighting is disclosed for simulating stage lights and coordination of virtual stage lights with real stage lights for live and recorded performances,” abstract).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Mcnelley to the combination of Islam and Iwase. The motivation would have been “to appear ultra-realistic” (McNelley, para. 399).
Conclusion
Applicant's amendment necessitated any new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan McCulley whose telephone number is (571)270-3754. The examiner can normally be reached Monday through Friday, 8:00am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN MCCULLEY/Primary Examiner, Art Unit 2611
Prosecution Insights