Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed on 15 March, 2024.
Response to Amendment
The amendment filed 13 February, 2026 has been entered.
The amendment of claims 1 – 5, 7, 8, and 10 - 12 has been acknowledged.
The cancellation of claim 9 has been acknowledged.
The addition of new claims 13 - 20 has been acknowledged.
Response to Arguments
Applicant’s arguments, see page 6 section “Claim interpretation under 35 U.S.C. 112(f)”, filed 13 February, 2026 with respect to the interpretation of claims 1, 7, 8, and 9 have been fully considered and are persuasive. The interpretation of claims 1, 7, 8, and 9 under 35 U.S.C. 112(f) have been withdrawn.
Applicant’s arguments, see page 6 section “Claim interpretation under 35 U.S.C. 112(f)”, filed 13 February, 2026 with respect to the rejection of claims 1, 7, 8, and 9 have been fully considered and are persuasive. The rejection of claims 1, 7, 8, and 9 under 35 U.S.C. 112(b) have been withdrawn.
Applicant’s arguments, see page 6 section “Claims 1 – 6 were provisionally rejected on the ground of non-statutory double patenting as being unpatentable over claims 9 – 14 of copending Appl. Ser. No. 18/690272”, filed 13 February, 2026 with respect to the rejection of claims 1 – 5 and 7 – 11 have been fully considered and are persuasive. The double patenting rejection of claims 1 – 5 and 7 – 11 has been withdrawn.
Applicant’s arguments, see page 7, section “Claims 1 – 12 were rejected under 35 U.S.C. 102(a)(2) as allegedly being anticipated by Yehezkel et al (U.S. Patent Publication No. 2014/0086554 A1, hereinafter “Yehezkel””, filed 13 February, 2026 with respect to the rejection of claims 1 – 12 have been fully considered and are persuasive. The rejection of claims 1 – 12 under 35 .U.S.C. 102(a)(2) has been withdrawn. However, the examiner disagrees with the applicant’s arguments that Yehezkel does not teach the newly amended claim 1 limitation of “extracting, from emotion data having a user emotion value for each scene of a moving image content, an emotion representative scene by selecting one or more scenes based on a statistical value calculated across the moving image content from degrees of a user emotion for the scenes”. Additionally, a new rejection has been made under 35 U.S.C. 103.
Applicant states on page 7 of the remarks filed 13 February, 2026 that Yehezkel does not disclose the newly amended claim 1 limitation of “extracting, from emotion data having a user emotion value for each scene of a moving image content, an emotion representative scene by selecting one or more scenes based on a statistical value calculated across the moving image content from degrees of a user emotion for the scenes”. Specifically, that Yehezkel merely detects visual cues, estimates emotion events, and indexes a video with metadata comprising estimated emotion events and timing data. The examiner respectfully disagrees. Yehezkel does teach a process of estimating the emotional state of the user based on detected features, this information is then compiled as metadata which is indexed with the video so as to enable efficient retrieval of information about the viewer’s emotional states at various times during the video (see ¶ 0019). This information is then used in conjunction with a summarization module which extracts video segments that are associated with an increased density of emotional events exceeding a threshold. Specific examples are detailed in ¶ 0022 “Intra-video search module 512 may be configured to search for segments, or short clips, within a video in response to a query from a user. For example, the user may request segments that are "funny" and the intra-video search module 512 may extract those segments based on the video index which identifies video frames that are associated with an emotional response related to humor.”. While the applicant claims they use “statistics” in their process and thus differ from Yehezkel, these statistics are not defined in the claim language. Under broadest reasonable interpretation the examiner believes segmenting a video based on emotional thresholds and then extracting video segments based on those thresholds falls within the broad concept of “based on a statistical value calculated across the moving image content from degrees of a user emotion for scenes”.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 5 and 17 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends
Claims 5 and 17 appear to recite subject matter that does not further limit the independent claim limitations of claim 1 and 13. Claims 1 and 13 recite “extracting, from emotion data having a user emotion value for each scene of a moving image content, an emotion representative scene by selecting one or more scenes based on a statistical value calculated across the moving image content from degrees of a user emotion for the scenes” whereas claims 5 and 17 recite “Extracting the emotion representative scene on a basis of a statistical value of the degree of the user emotion of the moving image content”. This does not appear to further limit the scope of the independent claim in a meaningful way.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yehezkel et al (U.S. Patent Publication No. 2014/0086554 A1, hereinafter “Yehezkel”) in view of Han et al (U.S. Patent Publication No. 2022/0012500 A1, hereinafter “Han”).
Regarding claim 1, Yehezkel teaches an information processing apparatus comprising:
A memory storing a program (¶ 0027: Embodiments of the methods described herein may be implemented in a system that includes one or more storage mediums having stored thereon, individually or in combination, instructions that when executed by one or more processors perform the methods.), and
At least one processor configured to execute the program operations comprising (¶ 0027: Here, the processor may include, for example, a system CPU (e.g., core processor) and/or programmable circuitry.):
extracting, from emotion data having a user emotion value for each scene of a moving image content (¶ 0013: Emotion based video processing module 108 may be configured to index the video with metadata that includes the estimated user emotional responses and associated timing information relative to frames of the video.), an emotion representative scene by selecting one or more scenes based on a statistical value calculated across the moving image content from degrees of a user emotion for the scenes (¶ 0020: Summarization module 508 may be configured to extract segments of a video, for example a video from video database 506, that are associated with an increased density of emotional events that exceed an emotional intensity threshold.); and
(Figure 4; ¶ 0017: Emotion manifold based estimation module 204 may be configured to estimate the emotional (or pseudo-emotional) state of the user 104 at a given time based on the mapping of the detected features from the feature space 302 to the property space (e.g., the lower dimension manifold 206, 304). This is illustrated in FIG. 4 which shows a plot of estimated emotions types 402 over time which may be associated with video frame numbers or frame time-stamps.).
Yehezkel does not explicitly teach causing a display of a time position of the extracted emotion representative scene together with an indication of at least one of a type of the user emotion or a degree of the user emotion associated with the extracted emotion representative scene.
However, Han does teach causing a display of a time position of (Figure 3; ¶ 0050: The device 1000 may photograph the user watching the first video and analyze a user's emotion from a captured image to generate a user emotion graph.; ¶ 0055: The display unit 1200 may display information processed in the device 1000. For example, the display unit 1200 may display a graphical user interface (GUI) for displaying a video played in the device 1000 or generating or displaying summary videos of the second video.; ¶ 0065: The user emotion graph may show a change of the emotions of the user watching the first video over time, and may be generated based on the scores of the emotions of the user, calculated for a frame or a scene of the video in which the user is photographed.; Examiner’s note: It would be obvious to one skilled in the art to display the user emotion graph on the display which is used to display various aspects of the emotion data from the invention of Han. As shown in Figure 3, both the graph and user emotion score pages appear as displayed information that would be output for viewing.).
Yehezkel and Han are considered to be analogous art as both pertain to monitoring and recording the emotions of a user watching video content. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system of video indexing with video reaction estimation (as taught by Yehezzkel) the device and method for generating a summary video (as taught by Han) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Han is able to rapidly provide information about video content preferred by users based on analyzing their individual emotions. (see ¶ 0005)
This motivation for the combination of Yehezkel and Han is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 2, the Yehezkel and Han combination teaches the information processing apparatus according to claim 1.
Additionally, Yehezkel teaches wherein the operations further comprise:
Extracting the emotion representative scene on a basis of a type of the user emotion (¶ 0022: Intra-video search module 512 may be configured to search for segments, or short clips, within a video in response to a query from a user. For example, the user may request segments that are "funny" and the intra-video search module 512 may extract those segments based on the video index which identifies video frames that are associated with an emotional response related to humor.).
Regarding claim 3, the Yehezkel and Han combination teaches the information processing apparatus according to claim 1.
Additionally, Yehezkel teaches wherein the operations further comprise:
Extracting the emotion representative scene on a basis of a degree of the user emotion (¶ 0021: Summarization module 508 may be configured to extract segments of a video, for example a video from video database 506, that are associated with an increased density of emotional events that exceed an emotional intensity threshold.).
Regarding claim 4, the Yehezkel and Han combination teaches the information processing apparatus according to claim 3.
Additionally, Yehezkel teaches wherein the operations further comprise:
Extracting, as the emotion representative scene, a scene in which the degree of the user emotion exceeds a threshold (¶ 0021: Summarization module 508 may be configured to extract segments of a video, for example a video from video database 506, that are associated with an increased density of emotional events that exceed an emotional intensity threshold.).
Regarding claim 5, the Yehezkel and Han combination teaches the information processing apparatus according to claim 3.
Additionally, Yehezkel teaches wherein the operations further comprise:
Extracting the emotion representative scene on a basis of a statistical value of the degree of the user emotion of the moving image content (¶ 0021: Summarization module 508 may be configured to extract segments of a video, for example a video from video database 506, that are associated with an increased density of emotional events that exceed an emotional intensity threshold. (emphasis added)).
Regarding claim 6, the Yehezkel and Han combination teaches the information processing apparatus according to claim 5.
Additionally, Yehezkel teaches wherein the statistical value includes a maximum value, a sorting result, an average value, or a standard deviation value (¶ 0019: Estimated emotions, generated by emotion estimation module 106, are received by descriptor generation module 502 which may be configured to convert the estimated emotions into a descriptor that may include, for example, genre classification, identification of dominant emotions, measurement of duration of climax and anti-climax of emotions and/or measurement of time intervals between emotional peaks. Indexing module 504 may be configured to index or tag the video by integrating the descriptor information, which may be in the form of metadata, into the video. The indexing may enable the efficient retrieval of information; Examiner’s note: The system of Yehezkel identifies dominant emotions which are understood to be the emotion which is maximally expressed (maximum value) by a user for a given scene.).
Regarding claim 7, the Yehezkel and Han combination teaches the information processing apparatus according to claim 1.
Additionally, Yehezkel teaches wherein the operations further comprise:
reproducing the extracted emotion representative scene out of the moving image content (¶ 0020: Summarization module 508 may be configured to extract segments of a video, for example a video from video database 506, that are associated with an increased density of emotional events that exceed an emotional intensity threshold… Such extracted segments may be combined to provide a summary of the video that may be tailored to the user. The length of the summary may be adjusted based on a setting of the intensity threshold.).
Regarding claim 8, the Yehezkel and Han combination teaches the information processing apparatus according to claim 1.
Additionally, Yehezkel teaches wherein the operations further comprise:
Extracting the extracted emotion representative scene out of the moving image content to generate a new moving image content (¶ 0020: Such extracted segments may be combined to provide a summary of the video that may be tailored to the user. The length of the summary may be adjusted based on a setting of the intensity threshold.).
Regarding claim 10, the Yehezkel and Han combination teaches the information processing apparatus according to claim 1.
Additionally, Yehezkel teaches wherein the operations further comprise:
Causing a display of a type and a degree of the user emotion in the extracted emotion representative scene at a time position corresponding to the extracted emotion representative scene on a time-axis slide bar corresponding to the moving image content (Figure 4; ¶ 0017: Emotion manifold based estimation module 204 may be configured to estimate the emotional (or pseudo-emotional) state of the user 104 at a given time based on the mapping of the detected features from the feature space 302 to the property space (e.g., the lower dimension manifold 206, 304). This is illustrated in FIG. 4 which shows a plot of estimated emotions types 402 over time which may be associated with video frame numbers or frame time-stamps.; ¶ 0019: Estimated emotions, generated by emotion estimation module 106, are received by descriptor generation module 502 which may be configured to convert the estimated emotions into a descriptor that may include, for example, genre classification, identification of dominant emotions, measurement of duration of climax and anti-climax of emotions and/or measurement of time intervals between emotional peaks. Indexing module 504 may be configured to index or tag the video by integrating the descriptor information, which may be in the form of metadata, into the video. The indexing may enable the efficient retrieval of information.; ¶ 0026: At operation 640, the video is indexed with metadata, the metadata including the estimated emotion events and timing data associated with the estimated emotion events. The timing data may be related to frame numbers associated with the video stream or the elapsed time relative to the beginning of the video or relative to any other suitable reference time.; Examiner’s note: It would be obvious to one skilled in the art to display the user emotion graph of figured 4 on the display which is used to display the segmented videos to the user.).
Regarding claim 11, the Yehezkel and Han combination teaches the information processing apparatus according to claim 1.
Additionally, Yehezkel teaches wherein the operations further comprise:
Causing a display of the type of the user emotion with a mark (¶ 0019: Estimated emotions, generated by emotion estimation module 106, are received by descriptor generation module 502 which may be configured to convert the estimated emotions into a descriptor that may include, for example, genre classification, identification of dominant emotions, measurement of duration of climax and anti-climax of emotions and/or measurement of time intervals between emotional peaks. Indexing module 504 may be configured to index or tag the video by integrating the descriptor information, which may be in the form of metadata, into the video. The indexing may enable the efficient retrieval of information.).
Regarding claim 12, the Yehezkel and Han combination teaches an information processing method comprising
a procedure of extracting an emotion representative scene on a basis of emotion data having a user emotion for each scene of a moving image content (¶ 0013: Emotion based video processing module 108 may be configured to index the video with metadata that includes the estimated user emotional responses and associated timing information relative to frames of the video. Emotion based video processing module 108 may further be configured to summarize, partition and/or search the video as well as perform additional operations, to be described in greater detail below, based on the indexing.).
Regarding claim 13, claim 13 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Han’s further teaching on:
A non-transitory computer readable medium storing a program, the program being executable by a processor to perform operations (¶ 0013: According to a third aspect of the disclosure, a non-transitory computer-readable recording medium may have recorded thereon a program which, when executed by a computer, performs the method according to the first aspect of the disclosure.)…
Regarding claim 14, claim 14 has been analyzed with regard to respective claim 2 and is rejected for the same reasons of obviousness as used above.
Regarding claim 15, claim 15 has been analyzed with regard to respective claim 3 and is rejected for the same reasons of obviousness as used above.
Regarding claim 16, claim 16 has been analyzed with regard to respective claim 4 and is rejected for the same reasons of obviousness as used above.
Regarding claim 17, claim 17 has been analyzed with regard to respective claim 5 and is rejected for the same reasons of obviousness as used above.
Regarding claim 18, claim 18 has been analyzed with regard to respective claim 6 and is rejected for the same reasons of obviousness as used above.
Regarding claim 19, claim 19 has been analyzed with regard to respective claim 7 and is rejected for the same reasons of obviousness as used above.
Regarding claim 20, claim 20 has been analyzed with regard to respective claim 8 and is rejected for the same reasons of obviousness as used above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Seo et al (U.S. Patent No. 9560411 B2) teaches a system of generating metadata for content based emotional information received from a client device relating to the content.
Mahapatra et al (U.S. Patent No. 12126791 B1) teaches a video monitoring system which monitors and records a person during a video and classifies behavioral information for each person in the video.
Sud et al (U.S. Patent Publication No. 2018/0314881 A1) teaches a system which captures images of users expressing emotions. A machine learnt algorithm then classifies each of the emotions expressed in the images.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW JONES whose telephone number is (703)756-4573. The examiner can normally be reached Monday - Friday 8:00-5:00 EST, off Every Other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW B. JONES/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667