DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 17-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claims 17-18 recites a “computer readable storage medium” and “a computer program product, wherein the computer program product..”. In the specification of the present application, the “computer readable medium and a computer program product..” are not excluding transmission media (see paragraph [001401], [00141]). Further the claim recites “a computer readable storage medium and a computer program product”, and the specification fails to provide a definition for that term. It also does not provide any indication that such storage medium is non-transitory. Thus, the broadest, reasonable interpretation of “a computer readable storage medium and a computer program product”” encompasses non-statutory subject matter (transmission media) that is unpatentable under 35 U.S.C. 101.
Accordingly, claims 17-18 fail to recite statutory subject matter under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5-8 and 12-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiaze et al. (CN112464019A) in view of Dreyfuss et al. (US Patent Publication No. 2008/0314228).
Regarding claim 1, Jiaze discloses a sound effect display method, comprising [see: receiving a first operation that is input to a target audio file [see para. 0017, 0018; the audio parameters include playback speed, and the target effect corresponding to the playback speed is a marquee effect. On the playback interface of the target audio, the target effect displayed includes: on the playback interface of the target audio, a marquee effect is displayed, with the scrolling speed changing according to the playback speed of the target audio; which corresponds to selection or playback audio file to initiate the sound affect display];
and displaying a sound effect display interface corresponding to the target audio file in response to the first operation [see para. 0133, 0134; By displaying special effects that change with the audio parameters, users can enjoy the audio while also watching the special effects that reflect the changes in the audio, enriching the display content during audio playback, providing users with a rich visual experience, and improving the user experience; which corresponds to displaying an audio visualization interface corresponding to the selected audio file], wherein the sound effect display interface comprises a cover image corresponding to the target audio file [see para. 0028-0031; Based on the captured user image, the corresponding shadow effect is generated. On the playback interface of the target audio, a shadow effect with a shaking frequency that changes with the playback speed is displayed which corresponds to displaying media related visual content such as a cover image associated with the audio file] and a plurality of sound wave columns arranged along an edge of the cover image [see para. 0220, 0221; stereo sound acquisition or noise reduction, multiple microphones can be used, which are set in different parts of the terminal. The speaker is used to convert electrical signals from processor or radio frequency circuit into sound waves; which corresponds to generate and display waveform visualizations comprising bars like sound wave elements based on audio attribute]; however, Jiaze fails to explicitly teach the cover image is scaled up or scaled down according to a preset sound attribute of the target audio file, and the plurality of sound wave columns extend towards an area where the cover image is located along a direction perpendicular to the edge of the cover image according to the preset sound attribute of the target audio file to form a sound wave waveform corresponding to the preset sound attribute of the target audio file.
Dreyfuss discloses the cover image is scaled up or scaled down according to a preset sound attribute of the target audio file [see para. 0024, 0025; a creation and visualization of not only the event surface, but also of various aspects of the underlying superstructure of the music in a representational form on a visual display. The representation comprises visual characters or sprites made of simple or complex two- or three-dimensional geometric shapes that move in time according to instructions from metadata synchronized to the actual musical recording, where the metadata can be derived from actual audio files, data files, such as MIDI files, and data created by an artist who is interacting, via a user interface, with the music using the inventive tool; which corresponds to real time visualization where audio attribute graphics amplitude scale sizes and forming waveform], and the plurality of sound wave columns extend towards an area where the cover image is located along a direction perpendicular to the edge of the cover image according to the preset sound attribute of the target audio file to form a sound wave waveform corresponding to the preset sound attribute of the target audio file [see para. 0027-0030 and figures 4A-4M; Factors affecting the specific visual form of a character depends on parameters such as pitch (both absolute and relative to the key area of the harmonic surface), dynamic volume, and salience. This visual representation correspond to the actual structure of the music as heard, where simultaneous harmonic events are parsed into individual notes which are then tied together into a rhythmic stream of linked events providing themes, melodies, bass figures, and other musical structures and the "sonic" (audiological/waveform-based) are then synchronized by an artist to the digital audio recording of the musical piece to drive the visualization; it is relevant to forming a waveform shape using soundwave elements driven by sound attributes mapping for scaling].
It would have been obvious to one of an ordinary skill in the art, having the teachings of Jiaz and Dreyfuss before the affective filing date of the claimed invention to modify, the forming waveform shape using sound wave elements of Jiaz to include adjusting direction and geometric behavior of visual elements based on sound attributes, as taught by Dreyfuss.
One would have been motivated to make such a combination in order to predict variation in graphical layout to create an integrated sound affect display and to improve visual expressiveness of the sound effect display.
Regarding claim 5, Jiaze discloses wherein, before displaying the sound effect display interface, the method further comprises: obtaining an original image corresponding to the target audio file; obtaining a first color component, a second color component, and a third color component of each pixel point in the original image [see para. 0165; taking the audio parameter as the rhythm point intensity and the target effect corresponding to the rhythm point intensity as a flashing line effect as an example. The process of displaying target effects on the terminal includes: if the target audio is played to the rhythm point timestamp, the terminal displays a flashing line effect on the playback interface of the target audio, where the color of the flashing line changes according to the intensity of the rhythm point]; and offsetting the first color component and the second color component of each pixel point in the original image, respectively, to obtain the cover image corresponding to the target audio file, wherein the first color component and the second color component are different in an offset direction and/or an offset distance [see para. 0166; based on the intensity of the rhythm points of the target audio, a flashing line effect is generated in real time, with the color of the flashing line changing according to the intensity of the rhythm points. This provides users with an effect that reflects the intensity of the rhythm points, enriching the display content during audio playback; which corresponds to coordinate multiple visual elements using common audio-driven parameters].
Regarding claim 6, Dreyfuss discloses wherein a maximum extension length of the sound wave column is a preset length, and the cover image is scaled up and scaled down within a preset scaling range [see para. 0033, 0066; the idea being that at some level there is a constancy of the elements (shape, size, position, degree of opacity, movement, surface texture and surface motion, color, etc.)--minimal when great artistic expression is desired, and moderate to maximal when communication of consistent parameters is desired--but in any case defining a visual language for the underlying music and various mappings that performed, volume of the individual pitches mapped to the size of the object, and color to cue the viewer-listener into a number of shared parameters by, for example, grouping instruments of the same family to a shared color space].
Regarding claim 7, Dreyfuss discloses wherein a color of a first sound wave column among the plurality of sound wave columns is a third color, and a color of a second sound wave column among the plurality of sound wave columns is a fourth color; and a color of a sound wave column located between the first sound wave column and the second sound wave column among the plurality of sound wave columns is gradually varied from the third color to the fourth color [see para. 0026, 0027, 0066; Factors affecting the specific visual form of a character depends on parameters such as pitch (both absolute and relative to the key area of the harmonic surface), dynamic volume, and salience various mappings that performed, volume of the individual pitches could be mapped to the size of the object, and color implemented to cue the viewer-listener into a number of shared parameters by, for example, grouping instruments of the same family to a shared color space; which corresponds to mapping sound attributes to visual parameters].
Regarding claim 8, Jiaze discloses wherein, before displaying the sound effect display interface corresponding to the target audio file, the method further comprises: obtaining an average color value of pixel points of the cover image; converting the average color value into a color in an HSV color space to obtain a base color; and determining the third color, the fourth color, and a background color of the sound effect display interface according to the base color [see para. 0164-0165; The process of displaying target effects on the terminal includes: if the target audio is played to the rhythm point timestamp, the terminal displays a flashing line effect on the playback interface of the target audio, where the color of the flashing line changes according to the intensity of the rhythm point. For example, the stronger the rhythm intensity (drum beat) of the target audio, the brighter the flash effect will be, such as red].
Regarding claim 12, Dreyfuss discloses wherein the first sound wave column and the second sound wave column are sound wave columns having initial positions located on a target straight line, respectively; the target straight line is a straight line that passes through a geometric center of the cover image and has a preset angle with respect to a horizontal direction [see para. 0032, 0033; perhaps the only constraints imposed that note pitch corresponds with a vertical y-axis position on the display, and that the number of characters reflects the number of voices on the display, and all other parameters are assignable by the user. For works which are intended to be used as a teaching tool and to help viewers appreciate the underlying structure, more of the parameters or rules fixed so that the viewer can learn corresponding relationships. For example, one always define string section portions to be represented by the color red, whereas the brass section portions defined to be represented by the color yellow. The idea being that at some level there is a constancy of the elements (shape, size, position, degree of opacity, movement, surface texture and surface motion, color, etc.)--minimal when great artistic expression is desired, and moderate to maximal when communication of consistent parameters is desired; which corresponds to positioning visual elements].
Regarding claim 13; Dreyfuss discloses wherein the cover image is scaled up or scaled down according to the preset sound attribute of the target audio file, which comprises the case that: the cover image is scaled up or scaled down according to a scaling factor row corresponding to a scaling factor corresponding to an attribute value of the preset sound attribute of a key audio frame of the target audio file, wherein the scaling factor corresponding to the attribute value of the preset sound attribute of the key audio frame of the target audio file is determined according to a pre-established corresponding relationship between the scaling factor and the attribute value of the preset sound attribute[see para. 0024, 0025; a creation and visualization of not only the event surface, but also of various aspects of the underlying superstructure of the music in a representational form on a visual display. The representation comprises visual characters or sprites made of simple or complex two- or three-dimensional geometric shapes that move in time according to instructions from metadata synchronized to the actual musical recording, where the metadata can be derived from actual audio files, data files, such as MIDI files, and data created by an artist who is interacting, via a user interface, with the music using the inventive tool[ which corresponds to real time visualization where audio attribute graphics amplitude scale sizes and forming waveform].
Regarding claim 14, Dreyfuss discloses wherein the plurality of sound wave columns extend towards an area where the cover image is located along the direction perpendicular to the edge of the cover image according to the preset sound attribute of the target audio file, which comprises the case that: the plurality of sound wave columns extend towards the area where the cover image is located along the direction perpendicular to the edge of the cover image according to a determined extension length, wherein an extension length of each sound wave column is determined according to a waveform corresponding to an attribute value of the preset sound attribute of a key audio frame of the target audio file [see para. 0027-0030 and figures 4A-4M; Factors affecting the specific visual form of a character depends on parameters such as pitch (both absolute and relative to the key area of the harmonic surface), dynamic volume, and salience. This visual representation will therefore correspond to the actual structure of the music as heard, where simultaneous harmonic events are parsed into individual notes which are then tied together into a rhythmic stream of linked events providing themes, melodies, bass figures, and other musical structures and the "sonic" (audiological/waveform-based) are then synchronized by an artist to the digital audio recording of the musical piece to drive the visualization; it is relevant to forming a waveform shape using soundwave elements driven by sound attributes mapping for scaling].
Regarding claim 15, Jiaze discloses a terminal device, comprising [see para. 0092; terminal at least one of the following devices: smartphone, smartwatch, desktop computer, laptop computer]: receiving a first operation that is input to a target audio file [see para. 0017, 0018; the audio parameters include playback speed, and the target effect corresponding to the playback speed is a marquee effect. On the playback interface of the target audio, the target effect displayed includes: on the playback interface of the target audio, a marquee effect is displayed, with the scrolling speed changing according to the playback speed of the target audio; which corresponds to selection or playback audio file to initiate the sound affect display];
and displaying a sound effect display interface corresponding to the target audio file in response to the first operation [see para. 0133, 0134; By displaying special effects that change with the audio parameters, users can enjoy the audio while also watching the special effects that reflect the changes in the audio, enriching the display content during audio playback, providing users with a rich visual experience, and improving the user experience; which corresponds to displaying an audio visualization interface corresponding to the selected audio file], wherein the sound effect display interface comprises a cover image corresponding to the target audio file [see para. 0028-0031; Based on the captured user image, the corresponding shadow effect is generated. On the playback interface of the target audio, a shadow effect with a shaking frequency that changes with the playback speed is displayed which corresponds to displaying media related visual content such as a cover image associated with the audio file] and a plurality of sound wave columns arranged along an edge of the cover image [see para. 0220, 0221; stereo sound acquisition or noise reduction, multiple microphones can be used, which are set in different parts of the terminal. The speaker is used to convert electrical signals from processor or radio frequency circuit into sound waves; which corresponds to generate and display waveform visualizations comprising bars like sound wave elements based on audio attribute]; however, Jiaze fails to explicitly teach the cover image is scaled up or scaled down according to a preset sound attribute of the target audio file, and the plurality of sound wave columns extend towards an area where the cover image is located along a direction perpendicular to the edge of the cover image according to the preset sound attribute of the target audio file to form a sound wave waveform corresponding to the preset sound attribute of the target audio file.
Dreyfuss discloses the cover image is scaled up or scaled down according to a preset sound attribute of the target audio file [see para. 0024, 0025; a creation and visualization of not only the event surface, but also of various aspects of the underlying superstructure of the music in a representational form on a visual display. The representation comprises visual characters or sprites made of simple or complex two- or three-dimensional geometric shapes that move in time according to instructions from metadata synchronized to the actual musical recording, where the metadata can be derived from actual audio files, data files, such as MIDI files, and data created by an artist who is interacting, via a user interface, with the music using the inventive tool; which corresponds to real time visualization where audio attribute graphics amplitude scale sizes and forming waveform], and the plurality of sound wave columns extend towards an area where the cover image is located along a direction perpendicular to the edge of the cover image according to the preset sound attribute of the target audio file to form a sound wave waveform corresponding to the preset sound attribute of the target audio file [see para. 0027-0030 and figures 4A-4M; Factors affecting the specific visual form of a character depends on parameters such as pitch (both absolute and relative to the key area of the harmonic surface), dynamic volume, and salience. This visual representation correspond to the actual structure of the music as heard, where simultaneous harmonic events are parsed into individual notes which are then tied together into a rhythmic stream of linked events providing themes, melodies, bass figures, and other musical structures and the "sonic" (audiological/waveform-based) are then synchronized by an artist to the digital audio recording of the musical piece to drive the visualization; it is relevant to forming a waveform shape using soundwave elements driven by sound attributes mapping for scaling].
It would have been obvious to one of an ordinary skill in the art, having the teachings of Jiaz and Dreyfuss before the affective filing date of the claimed invention to modify, the forming waveform shape using sound wave elements of Jiaz to include adjusting direction and geometric behavior of visual elements based on sound attributes, as taught by Dreyfuss.
One would have been motivated to make such a combination in order to predict variation in graphical layout to create an integrated sound affect display and to improve visual expressiveness of the sound effect display.
Regarding claim 16, Jiaze discloses an electronic apparatus, comprising a memory and a processor, wherein the memory is configured for storing a computer program; and the processor is configured to enable the electronic apparatus to realize the sound effect display method upon the processor calling the computer program [see 0231; a computer-readable storage medium is also provided, such as a memory including program code that can be executed by a processor in a terminal to perform the audio playback method].
Regarding claim 17, Jiaze discloses a computer-readable storage medium, wherein a computer program is stored thereon, and the computer program enables a computing apparatus to realize the sound effect display method [see 0231; a computer-readable storage medium is also provided, such as a memory including program code that can be executed by a processor in a terminal to perform the audio playback method].
Regarding claim 18, Jiaze discloses a computer program product, wherein the computer program product enables a computer to realize the sound effect display method [see 0231; a computer-readable storage medium is also provided, such as a memory including program code that can be executed by a processor in a terminal to perform the audio playback method].
Regarding claim 19, Dreyfuss discloses wherein a color of a first sound wave column among the plurality of sound wave columns is a third color, and a color of a second sound wave column among the plurality of sound wave columns is a fourth color; and a color of a sound wave column located between the first sound wave column and the second sound wave column among the plurality of sound wave columns is gradually varied from the third color to the fourth color [see para. 0027-0030 and figures 4A-4M; Factors affecting the specific visual form of a character depends on parameters such as pitch (both absolute and relative to the key area of the harmonic surface), dynamic volume, and salience. This visual representation will therefore correspond to the actual structure of the music as heard, where simultaneous harmonic events are parsed into individual notes which are then tied together into a rhythmic stream of linked events providing themes, melodies, bass figures, and other musical structures and the "sonic" (audiological/waveform-based) are then synchronized by an artist to the digital audio recording of the musical piece to drive the visualization; it is relevant to forming a waveform shape using soundwave elements driven by sound attributes mapping for scaling].
Regarding claim 20, Jiaze discloses wherein, before displaying the sound effect display interface corresponding to the target audio file, the method further comprises: obtaining an average color value of pixel points of the cover image; converting the average color value into a color in an HSV color space to obtain a base color; and determining the third color, the fourth color, and a background color of the sound effect display interface according to the base color [see para. 0164-0165; The process of displaying target effects on the terminal includes: if the target audio is played to the rhythm point timestamp, the terminal displays a flashing line effect on the playback interface of the target audio, where the color of the flashing line changes according to the intensity of the rhythm point. For example, the stronger the rhythm intensity (drum beat) of the target audio, the brighter the flash effect will be, such as red].
Allowable Subject Matter
Claims 2-4 and 9-11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure (See PTO-892).
Family et al. (US 2020/0396560) discloses a spatial audio system, including a primary network connected speaker, including a plurality of sets of drivers, where each set of drivers is oriented in a different direction, a processor system, memory containing an audio player application, wherein the audio player application configures the processor system to obtain an audio source stream from an audio source via the network interface, spatially encode the audio source, decode the spatially encoded audio source to obtain driver inputs for the individual drivers in the plurality of sets of drivers, where the driver inputs cause the drivers to generate directional audio.
Perrodin et al. (US 8,751,933) discloses images representing video frames in a first region of a user interface can be displayed. An audio waveform representing audio associated with at least a portion of a video frame can be displayed. At least three audio waveform portions including (i) a first portion of the audio waveform that exceeds a first predetermined threshold, (ii) a second portion of the audio waveform below the first predetermined threshold and above a second predetermined threshold, and (iii) a third portion of the audio waveform below the second predetermined threshold can be graphically distinguished.
A reference to specific paragraphs, columns, pages, or figures in a cited prior art reference is not limited to preferred embodiments or any specific examples. It is well settled that a prior art reference, in its entirety, must be considered for all that it expressly teaches and fairly suggests to one having ordinary skill in the art. Stated differently, a prior art disclosure reading on a limitation of Applicant's claim cannot be ignored on the ground that other embodiments disclosed were instead cited. Therefore, the Examiner's citation to a specific portion of a single prior art reference is not intended to exclusively dictate, but rather, to demonstrate an exemplary disclosure commensurate with the specific limitations being addressed. In re Heck, 699 F.2d 1331, 1332-33,216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006,1009, 158 USPQ 275, 277 (CCPA 1968)). In re: Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir. 2005); In re Fritch, 972 F.2d 1260, 1264, 23 USPQ2d 1780, 1782 (Fed. Cir. 1992); Merck & Co. v. Biocraft Labs., Inc., 874 F.2d 804, 807, 10 USPQ2d 1843, 1846 (Fed. Cir. 1989); In re Fracalossi, 681 F.2d 792,794 n.1,215 USPQ 569, 570 n.1 (CCPA 1982); In re Lamberti, 545 F.2d 747, 750, 192 USPQ 278, 280 (CCPA 1976); In re Bozek, 416 F.2d 1385, 1390, 163 USPQ 545, 549 (CCPA 1969).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAO H NGUYEN whose telephone number is (571)272-4053. The examiner can normally be reached on Mon-Fri 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached on 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAO H NGUYEN/Primary Examiner, Art Unit 2171