DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed March 2nd, 2026 has been entered. Claims 1-18 remain pending in the application. Claims 1, 5, 7 and 8 have been amended. Claims 2-4 and 6 have not been amended. Claims 9-18 are newly added. Applicant’s amendments to the Claims have overcome each and every 112(b) rejection previously set forth in the Non-Final Office Action mailed December 1st, 2025.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 7-11 and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Ghost Together (How to add border to HTML image (img) tag using CSS style.) in view of Mangolin (A Multimodal Approach for Multi-Label Movie Genre Classification, 2020), Chandrashekar (US 20230018555 A1) and Stack Overflow (How to change border color within if statement?).
Regarding Claim 1, Ghost Together teaches a display method comprising:
Displaying a second image disposed along an outline of the first image outside the first image (0:16: demonstrates a first picture of a cat with a red border picture disposed outside and around it); and
Displaying a third image disposed along the outline of the first image outside the first image and having a color or a pattern different from the second image (0:50: demonstrates the first picture of a cat with a grey, dashed border disposed outside and around it)
Ghost Together does not teach acquiring content information including audio data indicating a content of a first image.
However, Mangolin teaches acquiring content information including audio data indicating a content of a first image (Section 4, Proposed Method: “As aforementioned in this work, we aim at exploring data from different modalities (i.e., audio, video/image, and text) to classify the genre of movie titles. Thus, we chose specific methods that could lead us to obtain the best benefit in terms of the classification performance of each of these movie title data sources. To better understand the proposal of this work, Figure 3 presents a general overview of it, considering: the data source preparation (Phase 1), feature extraction (Phase 2), compress (only in case of huge representations) (Phase 3), resampling (Phase 4), classification (Phase 5), and fusion of the predictions (Phase 6)”; Figure 3: “General overview of features and classifiers used for multimodal multi-label classification”; Figure 3 clearly demonstrates that audio data corresponding to an image is acquired, and is subsequently utilized for classifying a genre of the image/movie in which the audio was derived from. Notes: It is worth noting that the broadest reasonable interpretation of “content information” in relation to indicating a content of an image is determined to be any information indicating a content of a first image. Applicant’s use of “acquiring content information” in its broadest reasonable interpretation is deemed to be the retrieval of content information, wherein the generation of said content information is arbitrary and assumed. Audio classification is an example in the art of providing labels, and consequently indicating content, for images with corresponding audio, and provides clear insight into the current depth of what is considered well known in the art).
Ghost Together and Mangolin are considered analogous in the art with respect to context for images. A person ordinarily skilled in the art would appreciate that displaying a contextually relevant border around an image is a well-known practice. A common motivation for doing so is to improve the appeal and aesthetic of the image for a viewer.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the display of an image around a first image with the acquiring of content information including audio data for a first image; Doing so would yield the predictable result of a contextually relevant image disposed along an outline of a first image, improving the appeal and aesthetic of the first image.
Ghost Together as modified does not explicitly teach analyzing the audio data to detect a preset sound pattern.
However, Chandrashekar teaches analyzing audio data to detect a preset sound pattern (Paragraph [0039]: "convolutional neural networks (CNN) to separate the various audio signals (e.g., background music from people talking in a movie scene, etc.). For example, a deep learning model can be trained from pre-existing recorded content (i.e., classified dataset) with sounds that are classified (e.g., piano, crowds cheering, bombing, police sirens, guitar, piano, etc.)").
Ghost Together as modified and Chandrashekar are considered analogous in the art with respect to audio classification. A person having ordinary skill in the art would appreciate that analyzing audio samples to detect preset audio patterns for classifying and labeling audio data is well known in the art.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the acquiring of content information including audio data of Ghost Together as modified with the analyzing of audio data to detect a preset sound pattern; Doing so would yield the predictable result of indicating content of a first image through the classification of audio data.
Ghost Together as modified does not teach displaying a second image disposed along an outline of the first image outside the first image when the content information meets a first condition or a third image disposed along an outline of the first image outside the first image when the content information meets a second condition, where the first and second condition are based on the detected preset sound pattern.
However, Stack Overflow teaches a method and motivation for displaying different images depending on different conditions being fulfilled (rushikesh chaskar: “[you] can have a single Border widget with the color property calling a function which has these if else statements and setting it accordingly! Dont forget to add the assignment statement in a setstate() Function”, wherein the condition values checked are “Low”, “Normal”, and “High”; the referenced code block is also contained within the forum. Notes: The reference demonstrates that changing a border, or any shape, dependent on a condition value being fulfilled is obvious within the art, and that there is motivation for depicting different border images based on a label condition. Note that Ghost Together as modified teaches acquiring content information including audio data indicating the content of a first image via labels and classification, and hence the conditions dependent on the labels are inherently based on the detected preset sound pattern as observed in Ghost Together as modified).
Ghost Together as modified and Stack Overflow are considered analogous in the art with respect to contextual display of images based on acquired content information (labels). Programmatically displaying multiple border images that differ from one another is apparent in the art as demonstrated by Ghost Together. Conditional functions are a core concept in programming, and as demonstrated by Stack Overflow, conditionally generating and changing border images based on an associated tag attribute that is checked in an if statement is apparent in the art. A common motivation for conditionally generating and changing border images based on an associated label attribute is to allow the display of contextually relevant border images with a main image.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the display of an image disposed around the border of a first image with the conditional display of images based on condition (label) values; Doing so would yield the predictable result of allowing the conditional display of an image disposed around the border of a first image to improve the aesthetic and visual appeal of the first image.
Claim 7, being similar in scope to Claim 1, is rejected under the same rationale.
Claim 8, being similar in scope to Claim 1, is rejected under the same rationale.
Regarding Claim 2, the display method of Claim 1 is rejected over Ghost Together as modified. The subject matter of Claim 2 is similar in scope to Claim 1 with respect to a display system wherein the content information indicates a genre of the first image, the first condition is that the genre of the first image is a first genre, and the second condition is that the genre of the first image is a second genre which is different from the first genre (Section 4, Proposed Method: “As aforementioned in this work, we aim at exploring data from different modalities (i.e., audio, video/image, and text) to classify the genre of movie titles”; Refer to Figure 3. Notes: genre is considered in its broadest reasonable interpretation to be synonymous, or at the very least falling within the category of a label, for an image. As specified in the rejection of Claim 1, a label is classified as content information indicating a content of an image).
Claim 9, being similar in scope to Claim 2, is rejected under the same rationale.
Claim 14, being similar in scope to Claim 2, is rejected under the same rationale.
Regarding Claim 3, the display method of Claim 1 is rejected over Ghost Together as modified. The subject matter of Claim 3 is similar in scope to Claim 1 with respect to a display method wherein the first image is a part of a video that changes with time, the content information indicates a scene of the first image in the video, the first condition is that the scene of the first image is a first scene, and the second condition is that the scene of the first image is a second scene which is different from the first scene (Section 4, Proposed Method: “As aforementioned in this work, we aim at exploring data from different modalities (i.e., audio, video/image, and text) to classify the genre of movie titles”; Refer to Figure 3, which demonstrates that video frames (images) are derived from a movie. Notes: a video in its broadest reasonable interpretation is a collection of images played chronologically; whether or not a first image is derived from a video does not alter the method of Claim 1, considering that the method deals only with displaying a second or third image conditionally on the information content of a first image, which is arbitrarily obtained from the video. A scene is considered in its broadest reasonable interpretation to be a description of the image, and consequently falls within the category of a label, for an image. As specified in the rejection of Claim 1, a label is classified as content information indicating a content of an image).
Claim 10, being similar in scope to Claim 3, is rejected under the same rationale.
Claim 15, being similar in scope to Claim 3, is rejected under the same rationale.
Regarding Claim 4, the display method of Claim 1 is rejected over Ghost Together as modified.
The subject matter of Claim 4 is similar in scope to Claim 1 with respect to a display method further comprising:
Receiving an input designating a combination of one candidate condition among a plurality of candidate conditions that are candidates for the first condition and one candidate image among a plurality of candidate images that are candidates for the second image; and
receiving an input designating a combination of another candidate condition among the plurality of candidate conditions that are candidates for the second condition and another candidate image among the plurality of candidate images that are candidates for the third image,
wherein displaying the second image includes displaying the one candidate image as the second image when the content information meets the one candidate condition, and displaying the third image includes displaying the other candidate image as the third image when the content information meets the other candidate condition (Notes: “a plurality of candidate conditions” and “a plurality of candidate images” are interpreted within the broadest reasonable interpretation to simply be a collection of possible conditions and a collection of possible images, respectively. Therefore, candidate images are indistinct from the images previously introduced as a second and a third image, and candidate conditions are indistinct from the conditions previously introduced as a first and second condition. A “combination” of a candidate condition and a candidate image is implicit in displaying the candidate image in response to a candidate condition being fulfilled; namely, that the content information indicating the content of a first image triggers the condition; this is demonstrated by Stack Overflow in the rejection of Claim 1. In its broadest reasonable interpretation, “an input” in the context of the scope of the invention is any information that is used by the method to generate an output, with the output being generating a candidate image disposed along the outline of the first image of Claim 1. As demonstrated by Stack Overflow, satisfying a condition results in a specific border image being displayed; In other words, inputting the content information of the first image (established to encompass a label) into the conditional statement for comparison with the candidate condition results in the candidate image being displayed, thus designating the candidate condition and candidate image as a combination). Therefore, Claim 4 is analyzed under the same rationale.
Claim 11, being similar in scope to Claim 4, is rejected under the same rationale.
Claim 16, being similar in scope to Claim 4, is rejected under the same rationale.
Claims 5, 12 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Ghost Together (How to add border to HTML image (img) tag using CSS style.) in view of Mangolin (A Multimodal Approach for Multi-Label Movie Genre Classification, 2020), Chandrashekar (US 20230018555 A1) and Stack Overflow (How to change border color within if statement?), in further view of Online Tutorials (Awesome CSS Border Animation & Image Hover Effects | Html CSS @OnlineTutorialsYT. Youtube, 2022).
Regarding Claim 5, the display method according to Claim 4 is rejected over Ghost Together as modified.
Ghost Together as modified teaches a plurality of candidate images.
Ghost Together as modified does not teach candidate images that include at least two of first pattern images including a first pattern that blinks in a first cycle, second pattern images including a second pattern whose color changes in a second cycle, and third pattern images including a third pattern whose lighting part changes in a third cycle.
However, Online Tutorials teaches a display method wherein candidate images include at least two of first pattern images including a first pattern that blinks in a first cycle, second pattern images including a second pattern whose color changes in a second cycle, and third pattern images including a third pattern whose lighting part changes in a third cycle (5:39-8:00: Portion of first pattern images that is lit up changes in a circle pattern, demonstrating both that the pattern images blink in a first cycle, a lighting part that changes in a third cycle; second pattern images where the color of the pattern images also changes throughout).
Applying borders with changing components (lights, blinking, etc.) is apparent in the art, as demonstrated by Online Tutorials. Incorporating the techniques demonstrated by Online Tutorials to the base method of generating static borders of Ghost Together would not warrant a significant inventive step for one ordinarily skilled in the art. One would be motivated to do so to more effectively match the theme of the border with the main image currently being displayed, where said motivation is established as good thematic design for improving viewer experience.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have combined the display method anticipated by Ghost Together as modified with the method of generating borders with changing components of Online Tutorials; doing so would yield the predictable result of a display method capable of using a border image with changing components to more effectively match the theme of the image currently being displayed.
Claim 12, being similar in scope to Claim 5, is rejected under the same rationale.
Claim 17, being similar in scope to Claim 5, is rejected under the same rationale.
Claims 6, 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ghost Together (How to add border to HTML image (img) tag using CSS style.) in view of Mangolin (A Multimodal Approach for Multi-Label Movie Genre Classification, 2020), Chandrashekar (US 20230018555 A1), and Stack Overflow (How to change border color within if statement?), in further view of GameLandVN (Lol PBE 2/26/2015 Update: End of Game Victory/Defeat New Animation. Youtube, 2015).
Regarding Claim 6, the display method according to Claim 1 is rejected over Ghost Together as modified.
Ghost Together as modified teaches displaying a second image disposed along a first image when the content information meets a first condition, and displaying a third image disposed along a first image when the content information meets a second condition.
Ghost Together does not teach a first and second display object with different appearances that indicate menu information, that are displayed depending on if a first or second condition are met, respectively.
However, GameLandVN teaches a display method wherein when the content information meets the first condition, displaying a first display object having a first appearance associated with the first condition and indicating menu information, and when the content information meets the second condition, displaying a second display object having a second appearance associated with the second condition and different from the first appearance and indicating the menu information (Whole Video: The depicted game, League of Legends, has one of two display objects with different appearances display on the screen, depending on whether the content information (win or lose) describes the condition/image (ally team base destroyed or enemy team base destroyed, respectively). The display objects differ in terms of theme and shape, with victory displayed with blue, and defeat displayed with red. Both display objects indicate menu information, namely the ability to “continue” and exit the game).
Displaying a display object, often in the form of an OSD (on screen display), is a common feature associated with software programs and methods involving user interaction with screens. As demonstrated by the video of GameLandVN, matching the theme of a display object containing a menu through color and shapes is well demonstrated within the art, and the motivation for doing so would be to visually integrate the display object to the images being displayed such that they are thematically appealing; this is a common design motivation in the art.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the display method that conditionally displays images based on acquired content information including audio data of Ghost Together as modified, with the conditional display of thematic display objects with menus of GameLandVN; doing so would yield the predictable result of allowing for a thematically fitting display object with a menu to be displayed along with border images that also thematically fit the main image currently being displayed.
Claim 13, being similar in scope to Claim 6, is rejected under the same rationale.
Claim 18, being similar in scope to Claim 6, is rejected under the same rationale.
Response to Arguments
Applicant’s arguments, see Applicant Remarks in Section II, pages 8-9, filed March 2nd, 2026, with respect to the rejection of Claim 1 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection is made in view of changes made to Claim 1 regarding the acquired content information including audio data, analyzing the audio data to detect a preset sound pattern, and the first and second condition being based at least in part on the detected sound pattern.
In response to the Applicant’s argument that Ghost Together (How to add border to HTML image (img) tag using CSS style.), in view of Label Studio (Getting Started with Image Classification) and Stack Overflow (How to change border color within if statement?) fail to teach Claim 1 as amended, the examiner agrees that Ghost Together, in view of Label Studio and Stack Overflow fail to teach the amended portions of Claim 1, regarding the acquired content information including audio data, analyzing the audio data to detect a preset sound pattern, and the first and second condition being based at least in part on the detected sound pattern.
In response to the Applicant’ argument that GameLandVN(Lol PBE 2/26/2015 Update: End of Game Victory/Defeat New Animation. Youtube, 2015) and Online Tutorials (Awesome CSS Border Animation & Image Hover Effects | Html CSS @OnlineTutorialsYT. Youtube, 2022) fail to teach the amended portions of Claim 1, the examiner also agrees that the auxiliary references GameLandVN and Online Tutorials similarly do not teach said amended portions.
Hence, a new ground for rejection is formed based on the introduction of the aforementioned new limitations in amended Claim 1.
Ghost Together teaches a display method comprising:
Displaying a second image disposed along an outline of the first image outside the first image (0:16: demonstrates a first picture of a cat with a red border picture disposed outside and around it); and
Displaying a third image disposed along the outline of the first image outside the first image and having a color or a pattern different from the second image (0:50: demonstrates the first picture of a cat with a grey, dashed border disposed outside and around it)
Ghost Together does not teach acquiring content information including audio data indicating a content of a first image.
However, Mangolin teaches acquiring content information including audio data indicating a content of a first image (refer to rejection of amended Claim 1).
Ghost Together as modified does not explicitly teach analyzing the audio data to detect a preset sound pattern.
However, Chandrashekar teaches analyzing audio data to detect a preset sound pattern (refer to rejection of amended Claim 1).
In combination, Ghost Together, Mangolin, and Chandrashekar teach the acquiring of content information (in the form of a label classification), including audio data, a well as analyzing audio data to detect a preset sound pattern (refer to motivation of combination in the rejection of amended Claim 1), and displaying an image disposed along an outline of the first image in accordance with the acquired content information. The examiner notes that the motivation for the combination of Ghost Together as modified and Stack Overflow does not change as a result of the amended portions of Claim 1, as Stack Overflow describes the use of conditions (labels) to conditionally display images; whether the conditions are based at least in part on preset sound pattern does not affect the conditional display of said labels.
Therefore, Amended Claim 1 is rejected over Ghost Together as modified by Mangolin, Chandrashekar, and Stack Overflow.
Amended Claims 7 and 8, being similar in scope to Amended Claim 1, are rejected under the same rationale.
Claims 2-6 and 9-18 are rejected over 35 U.S.C. 103 as detailed above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights