DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is responsive to: Application filed 18 Jul. 2024
Claims 1-20 are pending in this case. Claims 1, 11 and 17 are independent claims
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 17-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Babinowich et al. (Pub. No.: US 2023/0351706 A1; Filed: Apr 27, 2022) (hereinafter “Babinowich”)
Regarding independent claim 17, one or more computer-readable storage media storing instructions that, responsive to execution by a processing device, causes the processing device to perform operations comprising (0026; 0031):
initiating audio data acquisition based on a context of an event using generative artificial intelligence (AI) as implemented using one or more machine-learning models, the event involving interaction of a virtual object in a user interface with a depiction of a real-world physical environment captured by frames of a digital video (0006; 0018-0019; 0024; 0076;0077; 00115; 0161); and
presenting representations of a plurality of options of the audio data for display in a user interface that support user selection for output as part of the event (0024; 0076; 0077; 00115).
Regarding dependent claim 18, Babinowich disclose the one or more computer-readable storage media as described in claim 17, wherein the representations include textual descriptions of audio sources associated with respective said options (0024; 0076; 0077; 00115).
Regarding dependent claim 19, Babinowich disclose the one or more computer-readable storage media as described in claim 17, wherein the operations further comprise generating digital content including a selected option from the plurality of options of audio data (0024; 0076; 0077; 00115).
Regarding dependent claim 20, Babinowich disclose the one or more computer-readable storage media as described in claim 19, wherein the digital content includes the frames of the digital video and the virtual object (0046; 0078; 0108).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5 and 7-16 are rejected under 35 U.S.C. 103 as being unpatentable over Kwatra et al. (Pub. No.: US 2025/0023741 A1; Filed: Jul. 14, 2023) (hereinafter “Kwatra”) in view of Babinowich et al. (Pub. No.: US 2023/0351706 A1; Filed: Apr 27, 2022) (hereinafter “Babinowich”).
Regarding independent claim 1, Kwatra disclose a method comprising:
detecting, by a processing device, an event from one or more inputs defining interaction of a virtual object in a user interface with a depiction of a real-world physical environment captured by frames of a digital video (0015-0016; 0018; 0054-0055);
monitoring, by the processing device, a context of the event in the user interface (0003-0006; 0054-0057);
Kwatra does not expressly disclose generating, by the processing device, a prompt to initiate acquisition of audio data based on the context using one or more machine-learning models; and
presenting, by the processing device, the audio data acquired by the one or more machine- learning models for output via the user interface.
Babinowich teach generating, by the processing device, a prompt to initiate acquisition of audio data based on the context using one or more machine-learning models (0006; 0018-0019; 0024; 0076; 0077; 00115; 0161); and
presenting, by the processing device, the audio data acquired by the one or more machine- learning models for output via the user interface (0006; 0018-0019; 0024; 0076;0077; 00115; 0161).
Therefore before the effective filing date of the claims invention, it would have been obvious to one of ordinary skill in the art to combine Babinowich with Kwatra for the benefit of using images and videos to automatically generate virtual representations based on intuitively obtained information (0004).
Regarding dependent claim 2, Kwatra in view of Babinowich disclose the method as described in claim 1, wherein the frames of the digital video are captured by a digital camera of a computing device that includes the processing device and presenting is performed in the user interface as the frames are received using an audio output device (0005; 0009; 0046; 0024; 0076; 0077; 00115).
Regarding dependent claim 3, Kwatra in view of Babinowich disclose the method as described in claim 1, wherein the context defines an event (0069).
Regarding dependent claim 4, Kwatra in view of Babinowich disclose the method as described in claim 3, wherein the event type is: a tap on a real-world object depicted in the real-world physical environment of the user interface;
a tap on the virtual object depicted in the user interface;
movement of the virtual object on a surface;
a collision between the virtual object and another object;
an animation of the virtual object; or
appearance of the virtual object in the user interface (0067-0068; 0071).
Regarding dependent claim 5, Kwatra in view of Babinowich disclose the method as described in claim 1, wherein the prompt is configured solely using text as describing the context and the virtual object (0024; 0076; 0115).
Regarding dependent claim 7, Kwatra in view of Babinowich disclose the method as described in claim 1, wherein the one or more machine-learning models are configured to acquire the audio data using local recommendation, online retrieval, audio generation using an audio diffusion model, or audio transfer using text-based sound style transfer (0015; 0018-0019; 0076; 0077; 00115).
Regarding dependent claim 8, Kwatra in view of Babinowich disclose the method as described in claim 1, wherein the presenting includes presenting representations of a plurality of options of the audio data in the user interface that support user selection for output in the user interface in conjunction with the event (0024; 0076; 0077; 00115).
Regarding dependent claim 9, Kwatra in view of Babinowich disclose the method as described in claim 8, wherein the representations include textual descriptions of audio sources associated with respective said options (0024; 0076; 0115).
Regarding dependent claim 10, Kwatra disclose the method as described in claim 1, wherein the presenting includes presenting a collision warning of the virtual object with a depiction of a real-world object of the real-world physical environment in the user interface (0055).
Regarding independent claim 11, Kwatra disclose a computing device comprising:
a processing device (0004-0005); and
a computer-readable storage medium storing instructions that, responsive to execution by the processing device, causes the processing device to perform operations including (0005-0006; 0023):
monitoring a context of the event in the user interface (0003-0006; 0054-0057);
Kwatra does not expressly disclose detecting an event from one or more inputs, the event involving interaction of a subject with an object in a user interface;
generating a prompt to initiate acquisition of audio data based on the context using generative artificial intelligence (AI) as implemented using one or more machine-learning models; and
presenting the audio data acquired by the one or more machine-learning models for output via the user interface.
Babinowich teach detecting an event from one or more inputs, the event involving interaction of a subject with an object in a user interface (0082; 0115)
generating a prompt to initiate acquisition of audio data based on the context using generative artificial intelligence (AI) as implemented using one or more machine-learning models(0006; 0018-0019; 0024; 0076;0077; 00115; 0161); and
presenting the audio data acquired by the one or more machine-learning models for output via the user interface (0006; 0018-0019; 0024; 0076;0077; 00115; 0161).
Therefore before the effective filing date of the claims invention, it would have been obvious to one of ordinary skill in the art to combine Babinowich with Kwatra for the benefit of using images and videos to automatically generate virtual representations based on intuitively obtained information (0004).
Regarding dependent claim 12, Kwatra disclose the computing device as described in claim 11, wherein the user interface includes a depiction of a real-world physical environment (0036-0037; 0055).
Regarding dependent claim 13, Kwatra disclose the computing device as described in claim 11, wherein the input describes movement of the subject in relation to the object, the movement indicated through a user input as received via the user interface, and the presenting is performed in real time as the input is received describing the movement (0041; 0043).
Regarding dependent claim 14, Kwatra disclose the computing device as described in claim 11, wherein the subject is a virtual object and the object is captured of a real-world object in one or more frames of a digital video (0015-0016).
Regarding dependent claim 15, Kwatra disclose the computing device as described in claim 11, wherein the object is a virtual object and the subject is captured of a real-world object in one or more frames of a digital video (0015-0016).
Regarding dependent claim 16, Kwatra in view of Babinowich disclose the computing device as described in claim 11, wherein the one or more machine-learning models are configured to acquire the audio data using local recommendation, online retrieval, audio generation using an audio diffusion model, or audio transfer using text-based sound style transfer (0024; 0076; 0077; 00115).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Kwatra in view of Miller et al. (Pub. No.: US 2018/0059898 A1; Filed: Aug 24, 2016) (hereinafter “Miller”).
Regarding dependent claim 6, Kwatra does not expressly disclose the method as described in claim 1, wherein the generating of the prompt is performed by filling out a template based on the context and the virtual object.
Miller teach wherein the generating of the prompt is performed by filling out a template based on the context and the virtual object (0084).
Therefore before the effective filing date of the claims invention, it would have been obvious to one of ordinary skill in the art to combine Miller with Kwatra for the benefit developing virtual user experiences that go beyond the current proprietary hardware/software solutions (0002).
NOTE
It is noted that any citations to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the reference should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. See MPEP 2123.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES J DEBROW whose telephone number is (571)272-5768. The examiner can normally be reached on 09:00 - 06:00.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Bashore can be reached on 571-272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/James J Debrow/
Primary Patent Examiner
Art Unit 2174
571-272-5768
Prosecution Insights