DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is responsive to the amendments and remarks received 17 July 2026. Claims 1 - 20 are currently pending.
Claim Objections
Claim 11 is objected to because of the following informalities: Lines 5 - 6 of claim 11 recite, in part, “the sequence of frames; generating” which appears to contain a grammatical error and/or a minor informality. The Examiner suggests amending the claim to --the sequence of frames; and generating-- in order to improve the clarity and precision of the claim. Appropriate correction is required.
The objections to claims 1, 7, 9, 16 and 19, due to minor informalities, are hereby withdrawn in view of the amendments and remarks received 17 July 2026.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
The rejections to claims 2- 5 and 7 under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, are hereby withdrawn in view of the amendments and remarks received 17 July 2026.
Claim Rejections - 35 USC § 101
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1 - 9, 11 - 15 and 17 - 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, an abstract idea, without significantly more. The claims are directed towards generating information indicating inter-shot relationships between different shots of a primary content item and/or determining scene boundaries of the primary content item.
The claims recite, at a high level of generality, identifying, based on comparing images of sequential frames of the sequence of frames, a plurality of shot boundaries in the sequence of frames, generating, based on applying a first model to frames of shots of a plurality of shots defined by the plurality of shot boundaries, information indicating areas of attention in the frames, generating, based on applying a second model to the information indicating areas of attention, information indicating inter-shot relationships between frames of different shots of the plurality of shots, and determining, based on the information indicating inter-shot relationships, that one or more of the plurality of shot boundaries are scene boundaries in the primary content item.
The limitation of “identifying, based on comparing images of sequential frames of the sequence of frames, a plurality of shot boundaries in the sequence of frames”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind using observation, evaluation, judgment, and opinion. For example, the claimed identifying, based on comparing images of sequential frames of the sequence of frames, a plurality of shot boundaries in the sequence of frames encompasses a user viewing a sequence of frames of a content item and performing an evaluation by mentally detecting (identifying) transitions (shot boundaries) between shots in the sequence of frames. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, with or without the use of a physical aid such as pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III).
Similarly, the limitation of “generating, based on applying a first model to frames of shots of a plurality of shots defined by the plurality of shot boundaries, information indicating areas of attention in the frames”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind using observation, evaluation, judgment, and opinion. For example, the claimed generating, based on applying a first model to frames of shots of a plurality of shots defined by the plurality of shot boundaries, information indicating areas of attention in the frames encompasses a user observing frames of a plurality of shots and identifying one or more characteristics or regions of the frames that are unique, distinctive and/or interesting. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, with or without the use of a physical aid such as pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III).
Relatedly, the limitation of “generating, based on applying a second model to the information indicating areas of attention, information indicating inter-shot relationships between frames of different shots of the plurality of shots”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind using observation, evaluation, judgment, and opinion. For example, the claimed generating, based on applying a second model to the information indicating areas of attention, information indicating inter-shot relationships between frames of different shots of the plurality of shots encompasses a user observing and/or thinking about the one or more characteristics or regions of the frames that they identified as being unique, distinctive and/or interesting and determining similar unique, distinctive and/or interesting characteristics or regions of frames that appear in different shots. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, with or without the use of a physical aid such as pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III).
Likewise, the limitations of “determining, based on the information indicating inter-shot relationships, that one or more of the plurality of shot boundaries are scene boundaries in the primary content item” and/or “sending the inter-shot information to a prediction model for identifying scene boundaries within the content item”, as drafted, are processes that, under their broadest reasonable interpretations, cover performance of the limitations in the mind using observation, evaluation, judgment, and opinion. For example, the claimed determining, based on the information indicating inter-shot relationships, that one or more of the plurality of shot boundaries are scene boundaries in the primary content item and the claimed sending the inter-shot information to a prediction model for identifying scene boundaries within the content item encompass a user evaluating the unique, distinctive and/or interesting characteristics or regions identified in different shots that they determined to be similar to each other and deciding whether or not the different shots belong to the same scene or different scenes based on the unique, distinctive and/or interesting characteristics or regions identified in the different shots. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, with or without the use of a physical aid such as pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III).
This judicial exception is not integrated into a practical application. In particular, the claims recite additional elements of: “receiving… a sequence of frames of a primary content item”, “receiving… intra-shot information indicating areas of attention in frames of each of a plurality of shots of a content item” and “a computing device”.
The limitations of “receiving… a sequence of frames of a primary content item” and “receiving… intra-shot information indicating areas of attention in frames of each of a plurality of shots of a content item” are mere pre-solution activity, data gathering, recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP § 2106.05(g). In addition, all uses of the recited judicial exception require such data gathering, and, as such, this limitations do not impose any meaningful limits on the claims. These limitations amount to necessary data gathering. See MPEP § 2106.05.
Further, the limitation of “a computing device” is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. Furthermore, the claims as a whole merely describe how to generally “apply” the concept(s) of generating information indicating inter-shot relationships between different shots of a primary content item and/or determining scene boundaries of the primary content item in a computer environment. Simply implementing the abstract idea on a generic computer is not a practical application of the abstract idea. See MPEP § 2106.05(f).
Even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Accordingly, the claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of: “receiving… a sequence of frames of a primary content item” and “receiving… intra-shot information indicating areas of attention in frames of each of a plurality of shots of a content item” are mere pre-solution activity, data gathering, recited at a high level of generality, and are thus insignificant extra-solution activity. Furthermore, the additional element of: “a computing device” amounts to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Even when considered in combination, these additional elements represent mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept. The claims are not patent eligible.
Furthermore, with regards to dependent claims 12, 18 and 19, the limitations of “a self-attention model configured to identify areas of attention…”, “applying a self-attention model to the frames of the content item”, “providing output from the self-attention model to a gated state space model”, “applying a plurality of different self-attention models to the frames of the content item, wherein the plurality of different self-attention models are configured to focus on different types of visual features” and “wherein each of the plurality of different self-attention models is configured to provide output to a corresponding gated state space model”, as drafted, are processes that, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components, a self-attention model(s) and a gated state space model(s). That is, other than reciting a self-attention model(s) and a gated state space model(s), nothing in the claim elements preclude the steps from practically being performed in the mind. The Examiner asserts that the claims do not provide any details nor limit how the models operate or how their functions are performed, and the plain meanings of applying, providing and output[ting] encompass mental observations, evaluations, judgments, and/or opinions, e.g., a user observing frames of the content item and identifying one or more characteristics or regions that are unique, distinctive and/or interesting. Under their broadest reasonable interpretations when read in light of the specification, applying, providing and output[ting] encompass mental processes practically performed in the human mind by observation(s), evaluation(s), judgment(s) and/or opinion(s). For example, the claimed applying, providing and output[ting] encompass a user observing frames of the content item and identifying one or more characteristics or regions that are unique, distinctive and/or interesting. See MPEP § 2106.04(a)(2)(I) and § 2106.04(a)(2)(III). If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. The claims are not patent eligible.
Moreover, with regards to dependent claims 12, 18 and 19, the limitations of “a self-attention model configured to identify areas of attention…”, “applying a self-attention model to the frames of the content item”, “providing output from the self-attention model to a gated state space model”, “applying a plurality of different self-attention models to the frames of the content item, wherein the plurality of different self-attention models are configured to focus on different types of visual features” and “wherein each of the plurality of different self-attention models is configured to provide output to a corresponding gated state space model”, provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP § 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. Moreover, the aforementioned models are used to generally apply the abstract idea without placing any limits on how the aforementioned models function. See MPEP 2106.05(f). Additionally, the recitations of a self-attention model(s) and a gated state space model(s)” merely indicate a field of use or technological environment in which the judicial exception is performed. Although the additional elements of a self-attention model(s) and a gated state space model(s) limit the identified judicial exception generating information indicating inter-shot relationships between different shots of a primary content item and/or determining scene boundaries of the primary content item, these types of limitations merely confine the use of the abstract idea to a particular technological environment (machine learning) and thus fail to add an inventive concept to the claims. See MPEP 2106.05(h). The claims are not patent eligible.
In addition, with regards to dependent claims 2 - 9, 12 - 15 and 18 - 20, the Examiner asserts that claims 2 - 9, 12 - 15 and 18 - 20 are also directed to the abstract idea(s) of generating information indicating inter-shot relationships between different shots of a primary content item and/or determining scene boundaries of the primary content item and dependent claims 2 - 9, 12 - 15 and 18 - 20 merely further limit the abstract idea(s) claimed in independent claims 1, 11 and 17, for example by further identifying visual features to consider when generating the information indicating areas of attention and/or inter-shot information and/or by identifying outputting recited at a high level of generality corresponding to insignificant extra-solution activity. However, the Examiner asserts that a more detailed abstract idea remains an abstract idea and that none of the limitations of dependent claims 2 - 9, 12 - 15 and 18 - 20 considered as an ordered combination provide eligibility because taken as a whole the claims merely instruct the practitioner to apply the abstract idea using generic computer components. The claims are not eligible.
Response to Arguments
Applicant’s arguments with respect to claim(s) 12 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments filed 17 July 2026 have been fully considered but they are not persuasive.
On pages 8 - 10 of the remarks the Applicant’s Representative argues that instant claims 1 - 9, 11 - 15 and 17 - 20 are patent-eligible under 35 U.S.C. 101. The Applicant’s Representative argues that the aforementioned claims are patent-eligible at least because claim 1 “recites a particular improvement to the technological field of video scene segmentation, not an abstract idea implemented on a generic computer.” The Applicant’s Representative argues that the limitations of claim 1 recite a particular two-model architecture “that improves the underlying technology, providing a more accurate and more efficient basis for scene-boundary detection.” The Applicant’s Representative therefore argues that any alleged judicial exception of the aforementioned claims are integrated into a practical application and are thus patent-eligible under 35 U.S.C. 101. Furthermore, the Applicant’s Representative argues that the aforementioned claims are patent-eligible at least because the claims as a whole recite significantly more than any alleged judicial exception. The Applicant’s Representative argues that the “ordered combination recited in claim 1 … is a particular, non-conventional technical arrangement for video scene segmentation that improves the accuracy and efficiency of scene-boundary detection, rather than a generic instruction to apply an alleged abstract idea on a computer.” Therefore, the Applicant’s Representative argues that the aforementioned claims are patent-eligible under 35 U.S.C. 101.
The Examiner respectfully disagrees.
The Examiner asserts that instant claims 1 - 9, 11 - 15 and 17 - 20 are still found to be directed to a judicial exception, an abstract idea, without significantly
more. The Examiner asserts that the identified claims are not found to be directed to a particular improvement in the technological field of video scene segmentation nor a particular, non-conventional technical arrangement for video scene segmentation at least because, as discussed above in section 8 of the instant Office Action, but for the recitation of the generic computer components, the scope of the identified claims can be performed mentally and the generic computer components are merely used as a tool to perform the abstract idea of generating information indicating inter-shot relationships between different shots of a content item and/or determining scene boundaries. Furthermore, the Examiner asserts that just gathering and analyzing information and obtaining a result of the analysis, such as determining scene boundaries in a content item, does not provide a particular, non-conventional improvement in the technological field of video scene segmentation at least because the content item is not altered in any way, for example no segmentation or alteration of the primary content item is claimed. Moreover, the Examiner asserts that the identified claims do not recite a particular, non-conventional improvement in the technological field of video scene segmentation at least because video scene segmentation by its very nature relies on information from frames of a plurality of shots, i.e., intra-shot information, and determining when a change in and/or a relationship between such information from different shots, i.e., inter-shot information, signals a change in scenes. In addition, the Examiner asserts that the identified claims fail to recite the components or steps that provide the asserted technological improvement such that the full scopes of the broadest reasonable interpretation of the identified claims reflect the asserted technological improvement. Furthermore, the Examiner notes that a judicial exception alone cannot provide an improvement in a technology, see MPEP § 2106.05(a). Therefore, the Examiner asserts that the identified claims as a whole are not found to integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Accordingly, the Examiner asserts that the identified claims are still found to be rejected under 35 U.S.C. 101 as being directed to an abstract idea without significantly more.
On pages 11 - 12 of the remarks the Applicant’s Representative argues that the combination of Mun et al. and Chu et al. “fails to teach or suggest at least ‘applying a second model to the information indicating the areas of attention,’ and thus the second ‘generating’ limitation of claim 1, as amended.” The Applicant’s Representative argues that Mun et al. fail to teach or suggest the aforementioned disputed claim limitation at least because the second model of Mun et al., the contextual relation network (CRN), operates on shot embeddings which are shot-level representations, not “information indicating areas of attention in the frames”. Furthermore, the Applicant’s Representative argues that Chu et al. fail to teach or suggest the aforementioned disputed claim limitation at least because the second model of Chu et al., the partitioning neural network, operates on shot-level feature representations, which are not “information indicating areas of attention in the frames”. Therefore, the Applicant’s Representative argues that the combination of Mun et al. and Chu et al. fails to teach or suggest the aforementioned disputed claim limitation.
The Examiner respectfully disagrees.
The Examiner asserts that, at least, Mun et al. disclose the aforementioned disputed claim limitation, see at least page 3 section 3.1 - page 4 section 3.2, page 4 figures 2 and 3, page 6 section 3.4 - page 7 section 3.5, page 10 section 5, page 14 section A - section B and page 15 figure 5 of Mun et al. wherein they disclose that a “scene is a series of shots, which is a set of frames physically captured by the same camera during an uninterrupted period of time”, that “each shot s is given by a set of Nk key-frames”, that their model “consists of two main components: 1) shot encoder embedding a shot by capturing its spatio-temporal patterns, and 2) contextual relation network (CRN) capturing relationship between shots”, that “en is an encoding of shot sn by fENC while En = {en-K,…,en,…,en+K} and Cn = {cn-K,…,cn,…,cn+K} correspond to the input and output feature sequence for fCRN, respectively”, that “given an input sequence of shots Sn, we employ a scene boundary detection head hSBD to infer a prediction from the contextualized representation (cn) for the center shot sn”, that they present “a novel self-supervised framework for video scene segmentation, especially designed to learn contextual relationship between shots” and that for “the shot encoder, each shot is given by three key-frames (i.e., Nk = 3) and a shot encoding e is given by the averaged feature after inferring individual three key-frames using ResNet-50”.
The Examiner asserts that, as shown herein above and in the cited portions, Mun et al. disclose that a shot is given by a set of frames, that a shot encoder is individually applied to frames of a shot, sn, to extract an encoding of the shot, en, that captures its spatio-temporal patterns and that encodings of shots extracted by the shot encoder are input to a contextual relation network (CRN) to extract features that capture relationships between shots. The Examiner asserts that the encodings of shots extracted by the shot encoder of Mun et al. correspond to the claimed “information indicating areas of attention in the frames” at least because Mun et al. disclose that shots are composed of frames, that the shot encoder is individually applied to frames of shots to extract the encodings of the shots and that the extracted encodings of the shots capture spatio-temporal patterns of the shots, i.e., information indicating areas of attention in frames of the shots. Furthermore, the Examiner asserts that the claims do not recite nor require that the “information indicating areas of attention in the frames” corresponds to multiple distinct frame-level pieces of information nor that the second model operates on such information. Therefore, the Examiner asserts that the combination of Mun et al. and Chu et al. discloses the aforementioned disputed claim limitation.
On page 12 of the remarks the Applicant’s Representative argues that claims 1 and 11 are not rendered obvious by Mun et al. in view of Chu et al. at least because “neither reference’s second model generates ‘information indicating inter-shot relationships between frames of different shots of the plurality of shots,’ as claim 1 requires.” The Applicant’s Representative argues that Mun et al. fail to teach or suggest the aforementioned disputed claim limitation at least because the second model of Mun et al., the contextual relation network, “captures relationships between shots - that is, among the averaged shot-level embeddings - rather than ‘inter-shot relationships between frames of different shots,’ because Mun's averaging of each shot's key-frames leaves no frame-level information on which to base such relationships.” Furthermore, the Applicant’s Representative argues that Chu et al. fail to teach or suggest the aforementioned disputed claim limitation at least because Chu et al. “proceeds directly to a partitioning decision and never produces the intermediate ‘information indicating inter-shot relationships between frames of different shots of the plurality of shots’ that the second ‘generating’ limitation requires and on which the ‘determining’ limitation operates.” Therefore, the Applicant’s Representative argues that the combination of Mun et al. and Chu et al. fails to teach or suggest the aforementioned disputed claim limitation.
The Examiner respectfully disagrees.
The Examiner asserts that, at least, Mun et al. disclose the aforementioned disputed claim limitation, see at least page 3 section 3.1 - page 4 section 3.2, page 4 figures 2 and 3, page 6 section 3.4 - page 7 section 3.5, page 10 section 5, page 14 section A - section B and page 15 figure 5 of Mun et al. wherein they disclose that a “scene is a series of shots, which is a set of frames physically captured by the same camera during an uninterrupted period of time”, that “each shot s is given by a set of Nk key-frames”, that their model “consists of two main components: 1) shot encoder embedding a shot by capturing its spatio-temporal patterns, and 2) contextual relation network (CRN) capturing relationship between shots”, that “en is an encoding of shot sn by fENC while En = {en-K,…,en,…,en+K} and Cn = {cn-K,…,cn,…,cn+K} correspond to the input and output feature sequence for fCRN, respectively”, that “given an input sequence of shots Sn, we employ a scene boundary detection head hSBD to infer a prediction from the contextualized representation (cn) for the center shot sn”, that they present “a novel self-supervised framework for video scene segmentation, especially designed to learn contextual relationship between shots” and that for “the shot encoder, each shot is given by three key-frames (i.e., Nk = 3) and a shot encoding e is given by the averaged feature after inferring individual three key-frames using ResNet-50”.
The Examiner asserts that, as shown herein above and in the cited portions, Mun et al. disclose that a shot is given by a set of frames, that a shot encoder is individually applied to frames of a shot, sn, to extract an encoding of the shot, en, that captures its spatio-temporal patterns and that encodings of shots extracted by the shot encoder are input to a contextual relation network (CRN) to extract features that capture relationships between shots. The Examiner asserts that, at least, the contextualized representations, cn, of Mun et al. correspond to information indicating inter-shot relationships between frames of different shots at least because Mun et al. disclose that shots are composed of frames, that the shot encoder is individually applied to frames of shots to extract the encodings of the shots, that the extracted encodings of the shots capture spatio-temporal patterns of the shots, corresponding to information indicating areas of attention in frames of the shots, and that the encodings of the shots are input to the CRN to extract contextualized representations, cn, that capture relationships between different shots, i.e., inter-shot relationships between frames of different shots. Therefore, the Examiner asserts that the combination of Mun et al. and Chu et al. discloses the aforementioned disputed claim limitation.
On pages 13 - 14 of the remarks the Applicant’s Representative argues that neither Rao et al. nor Houlsby et al. teach or suggest “’generating, based on the intra-shot information, inter-shot information indicating visual relationships between frames of different shots of a same content item’ from ‘intra-shot information indicating areas of attention in frames of each of a plurality of shots of a content item,’ as recited by claim 17.” The Applicant’s Representative argues that Rao et al. fail to teach or suggest the aforementioned disputed claim limitation at least because the local-to-global framework of Rao et al. “does not generate ‘inter-shot information indicating visual relationships between frames of different shots of a same content item’ from ‘intra-shot information indicating areas of attention in frames of each of a plurality of shots of a content item.’" Furthermore, the Applicant’s Representative argues that Houlsby et al. fail to teach or suggest the aforementioned disputed claim limitation at least because in Houlsby et al. “self-attention is used to characterize individual images and is not applied to generate inter-shot information or to identify scene boundaries.” Therefore, the Applicant’s Representative argues that the combination of Rao et al. and Houlsby et al. fails to teach or suggest the aforementioned disputed claim limitation.
The Examiner respectfully disagrees.
The Examiner asserts that, at least, Rao et al. disclose the aforementioned disputed claim limitation, see at least page 2 left-hand column first-full paragraph - second-full paragraph, page 2 section 2 subsection “Scene understanding in long videos”, page 3 section 3.2 and page 4 section 4 - section 4.4 of Rao et al. wherein they disclose that they “can learn what constitute a boundary between scenes in a supervised way, and thus get the capability of differentiating between within-scene and cross-scene transitions. 2) We leverage the cues contained in multiple semantic elements, including place, cast, action, and audio, to identify the associations across shots”, that they “propose a framework considering both the relationship among shots locally and the relationship among scenes globally using multiple semantic elements, achieving much better segmentation results”, that “each shot is represented with multiple semantic aspects. We design a three-level model to incorporate different levels of contextual information, i.e. clip level (B), segment level (T) and movie level (G), based on the shot representation si”, that to “obtain semantic features for each shot si, we utilize 1) ResNet50… to get place features, 2) Faster-RCNN… to detect cast instances and… cast features, 3) TSN… to get action features, 4) NaverNet… to separate speech and background sound, and stft [25] to get their features respectively in a shot with 16K Hz sampling rate and 512 windowed signal length, and concatenate them to obtain audio features”, that “scene segmentation can be formulated as a binary classification problem on shot boundaries” and that they “propose a Boundary Network (BNet) to model the shot boundary. As shown in Equation 2, BNet, denoted as B, takes a clip of the movie with 2wb shots as input and outputs a boundary representation bi. Motivated by the intuition that a boundary representation should capture both the differences and the relations between the shots before and after, BNet consists of two branches, namely Bd and Br. Bd is modeled by two temporal convolution layers, each of them embeds the shots before and after the boundary respectively, following an inner product operation to calculate their differences. Br aims to capture the relations of the shots”.
The Examiner asserts that, as shown herein above and in the cited portions, Rao et al. disclose the aforementioned disputed claim limitation(s) at least because they disclose that a movie with n shots is represented as a shot sequence, that they utilize neural networks to extract semantic features from each of the shots and use the extracted semantic features to represent each of the shots, i.e., intra-shot information indicating areas of attention in frames of each of a plurality of shots of a content item, and that they generate boundary representations for each shot boundary of the movie that “capture both the differences and the relations between the shots before and after”, i.e., inter-shot information indicating visual relationships between frames of different shots of a same content item, based on the multiple extracted semantic features that represent each shot. Therefore, the Examiner asserts that Rao et al. in view of Houlsby et al. disclose the aforementioned disputed claim limitation(s).
On page 14 of the remarks the Applicant’s Representative argues that neither Rao et al. nor Houlsby et al. teach or suggest the arrangements of claims 18 or 19. In particular, the Applicant’s Representative argues that neither Rao et al. nor Houlsby et al. teach or suggest providing output(s) from a self-attention model(s) to a gated state space model(s) because the Bi-LSTM of Rao et al. “is a bidirectional long short-term memory recurrent neural network, which is not a gated state space model”.
The Examiner respectfully disagrees.
The Examiner asserts that a bidirectional long short-term memory recurrent neural network, a Bi-LSTM, corresponds to a gated state space model because one of ordinary skill in the art would understand that Bi-LSTMs comprise long short-term memory units that are composed of cells and gates and that the cells of the long short-term memory units store state values, states. See for example the pertinent prior art cited herein below in section 18 of the instant Office Action. Therefore, the Examiner asserts that Rao et al. disclose a gated state space model and that Rao et al. in view of Houlsby et al. disclose the arrangements of claims 18 and 19.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 7 - 11 and 14 - 16 are rejected under 35 U.S.C. 103 as being unpatentable over Mun et al., “Boundary-Aware Self-Supervised Learning For Video Scene Segmentation”, arXiv, arXiv:2201.05277v1, 14 Jan. 2022, pages 1 - 20, herein referred to as “Mun et al.”, in view of Chu et a l. U.S. Publication No. 2019/0147105 A1.
- With regards to claim 1, Mun et al. disclose a method (Mun et al., Pg. 1 Abstract, Pg. 3 § 3 - Pg. 4 § 3.2, Pg. 10 § 5) comprising: receiving a sequence of frames of a primary content item; (Mun et al., Pg. 1 Abstract, Pg. 1 § 1 - Pg. 2 Third-Full Paragraph, Pg. 3 § 3.1 - Pg. 4 § 3.2, Pg. 7 § 4.1) identifying a plurality of shot boundaries in the sequence of frames; (Mun et al., Pg. 1 Abstract, Pg. 1 § 1 - Pg. 2 Third-Full Paragraph, Pg. 3 § 3.1 - Pg. 4 § 3.2, Pg. 7 § 4.1) generating, based on applying a first model to frames of shots of a plurality of shots defined by the plurality of shot boundaries, information indicating areas of attention in the frames; (Mun et al., Pgs. 4 - 5 § 3.2, Pg. 4 Figs. 2 & 3, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1, Pg. 14 § A) generating, based on applying a second model to the information indicating the areas of attention, information indicating inter-shot relationships between frames of different shots of the plurality of shots; (Mun et al., Pg. 3 § 3.1 - Pg. 4 § 3.2, Pg. 4 Figs. 2 & 3, Pg. 6 § 3.4 - Pg. 7 § 4.1) and determining, based on the information indicating inter-shot relationships, that one or more of the plurality of shot boundaries are scene boundaries in the primary content item. (Mun et al., Pg. 1 Abstract, Pgs. 1 - 2 § 1, Pg. 3 § 3 - Pg. 4 § 3.2, Pg. 6 § 3.4 - Pg. 7 § 3.5) Mun et al. fail to disclose explicitly a computing device; and identifying, based on comparing images of sequential frames of the sequence of frames, a plurality of shot boundaries. Pertaining to analogous art, Chu et al. disclose a method (Chu et al., Abstract) comprising: receiving, by a computing device, (Chu et al., Abstract, Fig. 1A, Pg. 1 ¶ 0022 - 0023, Pg. 2 ¶ 0038 - Pg. 3 ¶ 0040, Pg. 3 ¶ 0048, Pg. 4 ¶ 0056, Pg. 5 ¶ 0068 - Pg. 6 ¶ 0075) a sequence of frames of a primary content item; (Chu et al., Pg. 1 ¶ 0012, Pg. 2 ¶ 0035, Pg. 2 ¶ 0039 - Pg. 3 ¶ 0040, Pg. 3 ¶ 0042 - 0046 and 0049) identifying, based on comparing images of sequential frames of the sequence of frames, a plurality of shot boundaries in the sequence of frames; (Chu et al., Figs. 1A - 2, Pg. 1 ¶ 0005 and 0008 - 0009, Pg. 3 ¶ 0042 and 0049) generating, based on applying a first model to frames of shots of a plurality of shots defined by the plurality of shot boundaries, information indicating areas of attention in the frames; (Chu et al., Abstract, Figs. 1A, 2 & 3, Pg. 1 ¶ 0010 - 0015 and 0019, Pg. 4 ¶ 0057 - Pg. 5 ¶ 0062) generating, based on applying a second model to the information indicating the areas of attention, information indicating inter-shot relationships between frames of different shots of the plurality of shots; (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005 and 0020 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055, Pg. 5 ¶ 0066 - 0067) and determining, based on the information indicating inter-shot relationships, that one or more of the plurality of shot boundaries are scene boundaries in the primary content item. (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005 and 0020 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055, Pg. 5 ¶ 0066 - 0067) Mun et al. and Chu et al. are combinable because they are both directed towards image processing methods for video scene segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Mun et al. with the teachings of Chu et al. A first modification would have been prompted in order to enhance the base device of Mun et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Utilizing a computing device to implement a method, as taught by Chu et al., would enhance the base device of Mun et al. by allowing for it to be implemented accurately and efficiently at high computational speed on computer architecture. Additionally, a second modification would have been prompted in order to enhance the base device of Mun et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Comparing images of sequential frames of the sequence of frames to identify a plurality of shot boundaries, as taught by Chu et al., would enhance the base device of Mun et al. by allowing for it to be utilized on videos that have not been previously partitioned into a plurality of shots so as to enable the base device of Mun et al. to be applied to an increased number of videos and thereby enhancing the overall appeal and usefulness of the base device of Mun et al. to potential end-users. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that a computing device would be utilized to perform the operations of the base device of Mun et al. so as to ensure that its operations are carried out accurately, efficiently and at high computational speed on computer architecture and in that images of sequential frames of a sequence of frames of a primary content item would be compared to identify a plurality of shot boundaries so as allow for the base device of Mun et al. to be utilized on an increased number of videos, such as videos that have not been previously partitioned into a plurality of shots, and thereby enhance the overall appeal and usefulness of the base device of Mun et al. to potential end-users. Therefore, it would have been obvious to combine Mun et al. with Chu et al. to obtain the invention as specified in claim 1.
- With regards to claim 7, Mun et al. in view of Chu et al. disclose the method of claim 1, further comprising applying a plurality of different first models to the frames of shots of the plurality of shots, (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 4 Fig. 3, Pgs. 7 - 8 § 4.1) wherein the plurality of different first models are configured to focus on different types of visual features; (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 4 Fig. 3, Pgs. 7 - 8 § 4.1) and wherein each of the plurality of different first models is configured to provide output to a corresponding second model. (Mun et al., Pg. 1 Abstract, Pg. 4 Fig. 3, Pgs. 6 - 7 § 3.4) In addition, analogous art Chu et al. disclose applying a plurality of different first models to the frames of shots of the plurality of shots, (Chu et al., Pg. 1 ¶ 0005 and 0010 - 0016, Pg. 4 ¶ 0058 - Pg. 5 ¶ 0062) wherein the plurality of different first models are configured to focus on different types of visual features; (Chu et al., Pg. 1 ¶ 0005 and 0010 - 0016, Pg. 4 ¶ 0058 - Pg. 5 ¶ 0062) and wherein each of the plurality of different first models is configured to provide output to a corresponding second model. (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005 and 0020 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055, Pg. 5 ¶ 0066 - 0067)
- With regards to claim 8, Mun et al. in view of Chu et al. disclose the method of claim 1, further comprising: using a first pair of a first model and a corresponding second model to focus on spatio-temporal patterns; (Mun et al., Pg. 1 Abstract, Pg. 4 § 3.2 - Pg. 6 § 3.4, Pg. 4 Fig. 3, Pgs. 7 - 8 § 4.1) and using a second pair of a first model and a corresponding second model to focus on contextualized features. (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 4 Fig. 3, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1) Mun et al. fail to disclose explicitly using a first pair of models to focus on faces; and using a second pair of models to focus on objects. Pertaining to analogous art, Chu et al. disclose using a first pair of a first model and a corresponding second model to focus on faces; (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005 and 0019 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055, Pg. 5 ¶ 0061 and 0066 - 0067) and using a second pair of a first model and a corresponding second model to focus on objects. (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005, 0015 and 0020 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055 and 0058, Pg. 5 ¶ 0061 and 0066 - 0067) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with additional teachings of Chu et al. This modification would have been prompted in order to enhance the combined base device of Mun et al. in view of Chu et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Utilizing different models to focus on faces and objects, respectively, as taught by Chu et al., would enhance the combined base device by improving its ability to identify and analyze various semantic content of interest in a video when attempting to locate moments with a maximum semantic transition. Furthermore, this modification would have been prompted by the teachings and suggestions of Mun et al. that their method attempts to divide a sequence so that semantics, such as places and characters, maximally changes, see at least page 2 figure 1, page 5 section 3.3 and page 5 figure 4 of Mun et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that different models focusing on faces and objects, respectively, would be utilized to improve the ability of the combined base device to locate moments in a video wherein semantics maximally change. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with additional teachings of Chu et al. to obtain the invention as specified in claim 8.
- With regards to claim 9, Mun et al. in view of Chu et al. disclose the method of claim 1, further comprising using the scene boundaries to generate different video segments of the primary content item. (Mun et al., Pg. 1 Abstract, Pg. 1 § 1 ¶ 1, Pg. 5 § 3.3, Pg. 5 Fig. 4, Pg. 7 § 3.5, Pg. 10 § 5, Pg. 20 Fig. 8) In addition, analogous art Chu et al. disclose using the scene boundaries to generate different video segments of the primary content item. (Chu et al., Abstract, Figs. 1A - 2, Pg. 1 ¶ 0005 - 0006 and 0020 - 0021, Pg. 2 ¶ 0024 - 0026, 0035 - 0036 and 0039, Pg. 3 ¶ 0044 - 0047, Pg. 4 ¶ 0052 - 0055, Pg. 5 ¶ 0065 - 0067)
- With regards to claim 10, Mun et al. in view of Chu et al. disclose the method of claim 1. Mun et al. fail to disclose explicitly adding a secondary content item to the primary content item at a location that is based on one of the scene boundaries; and causing transmission of a modified primary content item comprising the added secondary content item. Pertaining to analogous art, Chu et al. disclose adding a secondary content item to the primary content item at a location that is based on one of the scene boundaries; (Chu et al., Pg. 2 ¶ 0024) and a modified primary content item comprising the added secondary content item. (Chu et al., Pg. 2 ¶ 0024) Chu et al. fail to disclose explicitly causing transmission of the modified primary content item. However, the Examiner takes official notice of the fact that causing transmission of a modified primary content item is notoriously well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with additional teachings of Chu et al. This modification would have been prompted in order to enhance the combined base device of Mun et al. in view of Chu et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Adding a secondary content item to the primary content item at a location that is based on one of the scene boundaries to produce a modified primary content item, as taught by Chu et al., would enhance the combined base device by allowing for it to utilize the scene boundaries it determined in a number of related video processing functions and/or applications, such as video advertisement and/or information insertion applications, so as to expand the number and variety of applications in which the combined base device may be utilized and increase its overall appeal and usefulness to potential end-users. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that a secondary content item would be added to the primary content item at a location that is based on one of the scene boundaries to produce a modified primary content item so as to expand the number and variety of applications in which the combined base device may be utilized and increase its overall appeal and usefulness to potential end-users. In addition, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. to include causing transmission of the modified primary content item. This modification would have been prompted in order to enhance the ability of the combined base device of Mun et al. in view of Chu et al. with the notoriously well-known technique of transmitting a modified primary content item. Causing transmission of the modified primary content item would enhance the combined base device by enabling the modified primary to be easily and conveniently distributed to potential end-users for viewing thereby improving the overall appeal, usefulness and usability of the combined base device. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would be caused to transmit the modified primary content item so as to enable the modified primary to be easily and conveniently distributed to potential end-users for viewing. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with additional teachings of Chu et al. and the notoriously well-known technique of causing transmission of a primary content item to obtain the invention as specified in claim 10. The Examiner asserts that the common knowledge or well-known in the art statement is taken to be admitted prior art because applicant failed to traverse the examiner’s assertion of official notice.
- With regards to claim 11, Mun et al. disclose a method (Mun et al., Pg. 1 Abstract, Pg. 3 § 3 - Pg. 4 § 3.2, Pg. 10 § 5) comprising: receiving a sequence of frames of a primary content item; (Mun et al., Pg. 1 Abstract, Pg. 1 § 1 - Pg. 2 Third-Full Paragraph, Pg. 3 § 3.1 - Pg. 4 § 3.2, Pg. 7 § 4.1) identifying a plurality of shot boundaries in the sequence of frames; (Mun et al., Pg. 1 Abstract, Pg. 1 § 1 - Pg. 2 Third-Full Paragraph, Pg. 3 § 3.1 - Pg. 4 § 3.2, Pg. 7 § 4.1) generating information, based on applying a plurality of model pairs to frames of shots of a plurality of shots defined by the plurality of shot boundaries, (Mun et al., Pgs. 4 - 5 § 3.2, Pg. 4 Figs. 2 & 3, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1, Pg. 14 § A) wherein each model pair of the plurality of model pairs comprises: a first model configured to identify areas of attention within a frame; (Mun et al., Pgs. 4 - 5 § 3.2, Pg. 4 Figs. 2 & 3, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1, Pg. 14 § A) and a second model configured to determine, based on the areas of attention, inter-shot relationships between frames of different shots. (Mun et al., Pg. 3 § 3.1 - Pg. 4 § 3.2, Pg. 4 Figs. 2 & 3, Pg. 6 § 3.4 - Pg. 7 § 4.1) Mun et al. fail to disclose explicitly a computing device; and identifying, based on comparing images of sequential frames of the sequence of frames, a plurality of shot boundaries. Pertaining to analogous art, Chu et al. disclose a method (Chu et al., Abstract) comprising: receiving, by a computing device, (Chu et al., Abstract, Fig. 1A, Pg. 1 ¶ 0022 - 0023, Pg. 2 ¶ 0038 - Pg. 3 ¶ 0040, Pg. 3 ¶ 0048, Pg. 4 ¶ 0056, Pg. 5 ¶ 0068 - Pg. 6 ¶ 0075) a sequence of frames of a primary content item; (Chu et al., Pg. 1 ¶ 0012, Pg. 2 ¶ 0035, Pg. 2 ¶ 0039 - Pg. 3 ¶ 0040, Pg. 3 ¶ 0042 - 0046 and 0049) identifying, based on comparing images of sequential frames of the sequence of frames, a plurality of shot boundaries in the sequence of frames; (Chu et al., Figs. 1A - 2, Pg. 1 ¶ 0005 and 0008 - 0009, Pg. 3 ¶ 0042 and 0049) generating information, based on applying a plurality of model pairs to frames of shots of a plurality of shots defined by the plurality of shot boundaries, (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005, 0015 and 0019 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055 and 0058 - 0059, Pg. 5 ¶ 0061 and 0066 - 0067) wherein each model pair of the plurality of model pairs comprises: a first model configured to identify areas of attention within a frame; (Chu et al., Abstract, Figs. 1A, 2 & 3, Pg. 1 ¶ 0010 - 0015 and 0019, Pg. 4 ¶ 0057 - Pg. 5 ¶ 0062) and a second model configured to determine, based on the areas of attention, inter-shot relationships between frames of different shots. (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005 and 0020 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055, Pg. 5 ¶ 0066 - 0067) Mun et al. and Chu et al. are combinable because they are both directed towards image processing methods for video scene segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Mun et al. with the teachings of Chu et al. A first modification would have been prompted in order to enhance the base device of Mun et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Utilizing a computing device to implement a method, as taught by Chu et al., would enhance the base device of Mun et al. by allowing for it to be implemented accurately and efficiently at high computational speed on computer architecture. Additionally, a second modification would have been prompted in order to enhance the base device of Mun et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Comparing images of sequential frames of the sequence of frames to identify a plurality of shot boundaries, as taught by Chu et al., would enhance the base device of Mun et al. by allowing for it to be utilized on videos that have not been previously partitioned into a plurality of shots so as to enable the base device of Mun et al. to be applied to an increased number of videos and thereby enhancing the overall appeal and usefulness of the base device of Mun et al. to potential end-users. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that a computing device would be utilized to perform the operations of the base device of Mun et al. so as to ensure that its operations are carried out accurately, efficiently and at high computational speed on computer architecture and in that images of sequential frames of a sequence of frames of a primary content item would be compared to identify a plurality of shot boundaries so as allow for the base device of Mun et al. to be utilized on an increased number of videos, such as videos that have not been previously partitioned into a plurality of shots, and thereby enhance the overall appeal and usefulness of the base device of Mun et al. to potential end-users. Therefore, it would have been obvious to combine Mun et al. with Chu et al. to obtain the invention as specified in claim 11.
- With regards to claim 14, Mun et al. in view of Chu et al. disclose the method of claim 11, wherein the plurality of model pairs are configured to focus on different types of visual features. (Mun et al., Pg. 1 Abstract, Pg. 4 § 3.2 - Pg. 6 § 3.4, Pg. 4 Fig. 3, Pgs. 7 - 8 § 4.1) In addition, analogous art Chu et al. disclose wherein the plurality of model pairs are configured to focus on different types of visual features. (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005, 0015 and 0019 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055 and 0058 - 0059, Pg. 5 ¶ 0061 and 0066 - 0067)
- With regards to claim 15, Mun et al. in view of Chu et al. disclose the method of claim 11, further comprising: using a first model pair to focus on spatio-temporal patterns; (Mun et al., Pg. 1 Abstract, Pg. 4 § 3.2 - Pg. 6 § 3.4, Pg. 4 Fig. 3, Pgs. 7 - 8 § 4.1) and using a second model pair to focus on contextualized features. (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 4 Fig. 3, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1) Mun et al. fail to disclose explicitly using a first model pair to focus on faces; and using a second model pair to focus on objects. Pertaining to analogous art, Chu et al. disclose using a first model pair to focus on faces; (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005 and 0019 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055, Pg. 5 ¶ 0061 and 0066 - 0067) and using a second model pair to focus on objects. (Chu et al., Abstract, Figs. 1A & 2, Pg. 1 ¶ 0005, 0015 and 0020 - 0021, Pg. 2 ¶ 0026 and 0036, Pg. 3 ¶ 0044 - 0046, Pg. 4 ¶ 0051 - 0055 and 0058, Pg. 5 ¶ 0061 and 0066 - 0067) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with additional teachings of Chu et al. This modification would have been prompted in order to enhance the combined base device of Mun et al. in view of Chu et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Utilizing different models to focus on faces and objects, respectively, as taught by Chu et al., would enhance the combined base device by improving its ability to identify and analyze various semantic content of interest in a video when attempting to locate moments with a maximum semantic transition. Furthermore, this modification would have been prompted by the teachings and suggestions of Mun et al. that their method attempts to divide a sequence so that semantics, such as places and characters, maximally changes, see at least page 2 figure 1, page 5 section 3.3 and page 5 figure 4 of Mun et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that different models focusing on faces and objects, respectively, would be utilized to improve the ability of the combined base device to locate moments in a video wherein semantics maximally change. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with additional teachings of Chu et al. to obtain the invention as specified in claim 15.
- With regards to claim 16, Mun et al. in view of Chu et al. disclose the method of claim 11. Mun et al. fail to disclose explicitly adding a secondary content item to the primary content item at a location that is based on a scene boundary that is determined based on the inter-shot relationships determined by second models of the plurality of model pairs; and causing transmission of a modified primary content item comprising the added secondary content item. Pertaining to analogous art, Chu et al. disclose adding a secondary content item to the primary content item at a location that is based on a scene boundary that is determined based on the inter-shot relationships determined by second models of the plurality of model pairs; (Chu et al., Pg. 2 ¶ 0024) and a modified primary content item comprising the added secondary content item. (Chu et al., Pg. 2 ¶ 0024) Chu et al. fail to disclose explicitly causing transmission of the modified primary content item. However, the Examiner takes official notice of the fact that causing transmission of a modified primary content item is notoriously well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with additional teachings of Chu et al. This modification would have been prompted in order to enhance the combined base device of Mun et al. in view of Chu et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Adding a secondary content item to the primary content item at a location that is based on one of the scene boundaries to produce a modified primary content item, as taught by Chu et al., would enhance the combined base device by allowing for it to utilize the scene boundaries it determined in a number of related video processing functions and/or applications, such as video advertisement and/or information insertion applications, so as to expand the number and variety of applications in which the combined base device may be utilized and increase its overall appeal and usefulness to potential end-users. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that a secondary content item would be added to the primary content item at a location that is based on one of the scene boundaries to produce a modified primary content item so as to expand the number and variety of applications in which the combined base device may be utilized and increase its overall appeal and usefulness to potential end-users. In addition, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. to include causing transmission of the modified primary content item. This modification would have been prompted in order to enhance the ability of the combined base device of Mun et al. in view of Chu et al. with the notoriously well-known technique of transmitting a modified primary content item. Causing transmission of the modified primary content item would enhance the combined base device by enabling the modified primary to be easily and conveniently distributed to potential end-users for viewing thereby improving the overall appeal, usefulness and usability of the combined base device. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would be caused to transmit the modified primary content item so as to enable the modified primary to be easily and conveniently distributed to potential end-users for viewing. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with additional teachings of Chu et al. and the notoriously well-known technique of causing transmission of a primary content item to obtain the invention as specified in claim 16. The Examiner asserts that the common knowledge or well-known in the art statement is taken to be admitted prior art because applicant failed to traverse the examiner’s assertion of official notice.
Claims 2 - 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Mun et al., “Boundary-Aware Self-Supervised Learning For Video Scene Segmentation”, arXiv, arXiv:2201.05277v1, 14 Jan. 2022, pages 1 - 20, herein referred to as “Mun et al.”, in view of Chu et a l. U.S. Publication No. 2019/0147105 A1 as applied to claims 1 and 11 above, and further in view of Li et al. U.S. Patent No. 12,008,788.
- With regards to claim 2, Mun et al. in view of Chu et al. disclose the method of claim 1, wherein the applying the first model to frames of shots of the plurality of shots comprises: identifying visual similarities. (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1) Mun et al. fail to disclose explicitly dividing a frame into a plurality of patches; and identifying visual similarities between the patches of the frame. Pertaining to analogous art, Li et al. disclose dividing a frame into a plurality of patches; (Li et al., Abstract, Col. 2 Lines 31 - 64, Col. 5 Lines 22 - 51, Col. 9 Lines 55 - 67) and identifying visual similarities between the patches of the frame. (Li et al., Col. 3 Lines 29 - 65, Col. 5 Lines 52 - 63, Col. 8 Line 44 - Col. 9 Line 7, Col. 10 Lines 10 - 42, Col. 11 Line 27 - Col. 12 Line 11) Mun et al. in view of Chu et al. and Li et al. are combinable because they are all directed towards image processing methods for video scene understanding applications. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with the teachings of Li et al. This modification would have been prompted in order to substitute the first model of Mun et al. for the vision transformer model of Li et al. The vision transformer model of Li et al. could be substituted in place of the first model of Mun et al. utilizing well-known techniques in the art and would likely yield predictable results, in that in the combination the video transformer model of Li et al. would be utilized to generate the representations of the shots. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the video transformer model of Li et al. would be utilized to generate the representations of the shots. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with Li et al. to obtain the invention as specified in claim 2.
- With regards to claim 3, Mun et al. in view of Chu et al. disclose the method of claim 1, wherein the applying the first model to frames of shots of the plurality of shots comprises: identifying visual similarities. (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1) Mun et al. fail to disclose explicitly dividing a frame into a plurality of patches; and identifying positional relationships between patches, of the frame, that comprise visual similarities. Pertaining to analogous art, Li et al. disclose dividing a frame into a plurality of patches; (Li et al., Abstract, Col. 2 Lines 31 - 64, Col. 5 Lines 22 - 51, Col. 9 Lines 55 - 67) and identifying positional relationships between patches, of the frame, that comprise visual similarities. (Li et al., Col. 3 Lines 29 - 65, Col. 5 Lines 52 - 63, Col. 6 Lines 54 - 63, Col. 7 Lines 50 - 63, Col. 8 Line 44 - Col. 9 Line 7, Col. 10 Lines 10 - 42, Col. 11 Line 27 - Col. 12 Line 11) Mun et al. in view of Chu et al. and Li et al. are combinable because they are all directed towards image processing methods for video scene understanding applications. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with the teachings of Li et al. This modification would have been prompted in order to substitute the first model of Mun et al. for the vision transformer model of Li et al. The vision transformer model of Li et al. could be substituted in place of the first model of Mun et al. utilizing well-known techniques in the art and would likely yield predictable results, in that in the combination the video transformer model of Li et al. would be utilized to generate the representations of the shots. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the video transformer model of Li et al. would be utilized to generate the representations of the shots. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with Li et al. to obtain the invention as specified in claim 3.
- With regards to claim 4, Mun et al. in view of Chu et al. disclose the method of claim 1, wherein the applying the first model to frames of shots of the plurality of shots comprises: identifying visual similarities between different frames in the first shot. (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1) Mun et al. fail to disclose explicitly dividing each frame, of a plurality of frames in a first shot, into a plurality of patches; and identifying visual similarities between patches of different frames. Pertaining to analogous art, Li et al. disclose dividing each frame, of a plurality of frames in a first shot, into a plurality of patches; (Li et al., Abstract, Col. 2 Lines 31 - 64, Col. 3 Lines 29 - 51, Col. 5 Lines 22 - 51, Col. 9 Lines 55 - 67) and identifying visual similarities between patches of different frames in the first shot. (Li et al., Col. 3 Lines 29 - 65, Col. 5 Lines 52 - 63, Col. 6 Lines 54 - 63, Col. 7 Lines 50 - 63, Col. 8 Line 44 - Col. 9 Line 7, Col. 10 Lines 10 - 42, Col. 11 Line 27 - Col. 12 Line 11) Mun et al. in view of Chu et al. and Li et al. are combinable because they are all directed towards image processing methods for video scene understanding applications. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with the teachings of Li et al. This modification would have been prompted in order to substitute the first model of Mun et al. for the vision transformer model of Li et al. The vision transformer model of Li et al. could be substituted in place of the first model of Mun et al. utilizing well-known techniques in the art and would likely yield predictable results, in that in the combination the video transformer model of Li et al. would be utilized to generate the representations of the shots. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the video transformer model of Li et al. would be utilized to generate the representations of the shots. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with Li et al. to obtain the invention as specified in claim 4.
- With regards to claim 5, Mun et al. in view of Chu et al. disclose the method of claim 1, wherein the applying the first model to frames of shots of the plurality of shots comprises: identifying representations that: are visually similar; (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1) and are of different frames in the first shot. (Mun et al., Pg. 1 Abstract, Pgs. 4 - 5 § 3.2, Pg. 6 § 3.4, Pgs. 7 - 8 § 4.1) Mun et al. fail to disclose explicitly dividing each frame, of a plurality of frames in a first shot, into a plurality of patches; and identifying positional relationships between patches that: are visually similar; and are of different frames in the first shot. Pertaining to analogous art, Li et al. disclose dividing each frame, of a plurality of frames in a first shot, into a plurality of patches; (Li et al., Abstract, Col. 2 Lines 31 - 64, Col. 3 Lines 29 - 51, Col. 5 Lines 22 - 51, Col. 9 Lines 55 - 67) and identifying positional relationships between patches that: are visually similar; and are of different frames in the first shot. (Li et al., Col. 3 Lines 29 - 65, Col. 5 Lines 52 - 63, Col. 6 Lines 54 - 63, Col. 7 Lines 50 - 63, Col. 8 Line 44 - Col. 9 Line 7, Col. 10 Lines 10 - 42, Col. 11 Line 27 - Col. 12 Line 11) Mun et al. in view of Chu et al. and Li et al. are combinable because they are all directed towards image processing methods for video scene understanding applications. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with the teachings of Li et al. This modification would have been prompted in order to substitute the first model of Mun et al. for the vision transformer model of Li et al. The vision transformer model of Li et al. could be substituted in place of the first model of Mun et al. utilizing well-known techniques in the art and would likely yield predictable results, in that in the combination the video transformer model of Li et al. would be utilized to generate the representations of the shots. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the video transformer model of Li et al. would be utilized to generate the representations of the shots. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with Li et al. to obtain the invention as specified in claim 5.
- With regards to claim 12, Mun et al. in view of Chu et al. disclose the method of claim 11. Mun et al. fail to disclose explicitly wherein the first model comprises a self-attention model configured to identify areas of attention among a plurality of patches divided from the frame. Pertaining to analogous art, Li et al. disclose wherein the first model comprises a self-attention model configured to identify areas of attention among a plurality of patches divided from the frame. (Li et al., Abstract, Figs. 2 - 4, 7 & 8, Col. 2 Lines 31 - 64, Col. 3 Lines 29 - 65, Col. 4 Lines 13 - 21, Col. 5 Lines 22 - 63, Col. 7 Line 50 - Col. 8 Line 3, Col. 8 Line 11 - Col. 9 Line 15, Col. 10 Lines 10 - 42, Col. 11 Line 27 - Col. 12 Line 11) Mun et al. in view of Chu et al. and Li et al. are combinable because they are all directed towards image processing methods for video scene understanding applications. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with the teachings of Li et al. This modification would have been prompted in order to substitute the first model of Mun et al. for the vision transformer model of Li et al. The vision transformer model of Li et al. could be substituted in place of the first model of Mun et al. utilizing well-known techniques in the art and would likely yield predictable results, in that in the combination the video transformer model of Li et al. would be utilized to generate the representations of the shots. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the video transformer model of Li et al. would be utilized to generate the representations of the shots. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with Li et al. to obtain the invention as specified in claim 12.
Claims 6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Mun et al., “Boundary-Aware Self-Supervised Learning For Video Scene Segmentation”, arXiv, arXiv:2201.05277v1, 14 Jan. 2022, pages 1 - 20, herein referred to as “Mun et al.”, in view of Chu et a l. U.S. Publication No. 2019/0147105 A1 as applied to claims 1 and 11 above, and further in view of Aoki et al. U.S. Publication No. 2009/0052783 A1.
- With regards to claim 6, Mun et al. in view of Chu et al. disclose the method of claim 1. Mun et al. fail to disclose explicitly using the second model to generate information indicating a positional relationship of a common object found in sequential frames of different shots. Pertaining to analogous art, Aoki et al. disclose using the second model to generate information indicating a positional relationship of a common object found in sequential frames of different shots. (Aoki et al., Abstract, Figs. 10 - 13, Pg. 2 ¶ 0032 - 0033, Pg. 3 ¶ 0038 - 0046, Pg. 4 ¶ 0051 - 0060, Pg. 5 ¶ 0072 - 0075) Mun et al. in view of Chu et al. and Aoki et al. are combinable because they are all directed towards image processing methods for video scene segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with the teachings of Aoki et al. This modification would have been prompted in order to enhance the combined base device of Mun et al. in view of Chu et al. with the well-known and applicable technique Aoki et al. applied to a comparable device. Generating information indicating a positional relationship of a common object found in sequential frames of different shots, as taught by Aoki et al., would enhance the combined base device by improving its ability to accurately and reliably identify similarities and differences between different shots of video since, in addition to the semantic content of the frames, the locations of the semantic content within the frames of the different shots would be analyzed when evaluating the different shots for scene boundaries. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that information indicating a positional relationship of a common object found in sequential frames of different shots would be generated so as to improve the ability of the combined base device to accurately and reliably identify similarities and differences between different shots of video since, in addition to the semantic content of the frames, the locations of the semantic content within the frames would be analyzed when evaluating the different shots for scene boundaries. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with Aoki et al. to obtain the invention as specified in claim 6.
- With regards to claim 13, Mun et al. in view of Chu et al. disclose the method of claim 11. Mun et al. fail to disclose explicitly wherein the second model is configured to generate information indicating a positional relationship of a common object found in sequential frames of different shots. Pertaining to analogous art, Aoki et al. disclose wherein the second model is configured to generate information indicating a positional relationship of a common object found in sequential frames of different shots. (Aoki et al., Abstract, Figs. 10 - 13, Pg. 2 ¶ 0032 - 0033, Pg. 3 ¶ 0038 - 0046, Pg. 4 ¶ 0051 - 0060, Pg. 5 ¶ 0072 - 0075) Mun et al. in view of Chu et al. and Aoki et al. are combinable because they are all directed towards image processing methods for video scene segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Mun et al. in view of Chu et al. with the teachings of Aoki et al. This modification would have been prompted in order to enhance the combined base device of Mun et al. in view of Chu et al. with the well-known and applicable technique Aoki et al. applied to a comparable device. Generating information indicating a positional relationship of a common object found in sequential frames of different shots, as taught by Aoki et al., would enhance the combined base device by improving its ability to accurately and reliably identify similarities and differences between different shots of video since, in addition to the semantic content of the frames, the locations of the semantic content within the frames of the different shots would be analyzed when evaluating the different shots for scene boundaries. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that information indicating a positional relationship of a common object found in sequential frames of different shots would be generated so as to improve the ability of the combined base device to accurately and reliably identify similarities and differences between different shots of video since, in addition to the semantic content of the frames, the locations of the semantic content within the frames would be analyzed when evaluating the different shots for scene boundaries. Therefore, it would have been obvious to combine Mun et al. in view of Chu et al. with Aoki et al. to obtain the invention as specified in claim 13
Claims 17 - 19 are rejected under 35 U.S.C. 103 as being unpatentable over Rao et al., “A Local-to-Global Approach to Multi-modal Movie Scene Segmentation”, arXiv, arXiv:2004.02678v3, 28 Apr. 2020, pages 1 - 10, herein referred to as “Rao et al.”, in view of Houlsby et a l. U.S. Publication No. 2022/0108478 A1.
- With regards to claim 17, Rao et al. disclose a method (Rao et al., Pg. 1 Abstract, Pg. 4 § 4, Pg. 8 § 6) comprising: receiving intra-shot information indicating areas of attention in frames of each of a plurality of shots of a content item; (Rao et al., Pg. 2 Left-Hand Column First-Full Paragraph - Second-Full Paragraph, Pg. 3 § 3.2, Pg. 4 § 4 - § 4.3) generating, based on the intra-shot information, inter-shot information indicating visual relationships between frames of different shots of a same content item; (Rao et al., Pg. 2 § 2 Subsection “Scene understanding in long videos”, Pg. 4 § 4 - § 4.4) and sending the inter-shot information to a prediction model for identifying scene boundaries within the content item. (Rao et al., Pg. 4 § 4 and § 4.2 - § 4.4, Pg. 5 Fig. 3) Rao et al. fail to disclose explicitly a computing device. Pertaining to analogous art, Houlsby et al. disclose receiving, by a computing device, (Houlsby et al., Abstract, Pg. 1 ¶ 0017 - Pg. 2 ¶ 0018, Pg. 5 ¶ 0063 - 0064, Pg. 8 ¶ 0101, Pg. 11 ¶ 0143 - 0146, Pg. 12 ¶ 0148 - 0151) information indicating areas of attention in frames of a content item. (Houlsby et al., Abstract, Pg. 2 ¶ 0022 - 0025, Pg. 4 ¶ 0054 - 008, Pg. 5 ¶ 0066 - Pg. 6 ¶ 0072, Pg. 6 ¶ 0078 - Pg. 7 ¶ 0083, Pg. 8 ¶ 0097 - 0099) Rao et al. and Houlsby et al. are combinable because they are both directed towards image and/or video processing methods that generate one or more representations that characterize an image or video. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rao et al. with the teachings of Houlsby et al. This modification would have been prompted in order to enhance the base device of Rao et al. with the well-known and applicable technique Houlsby et al. applied to a similar device. Utilizing a computing device to implement a method, as taught by Houlsby et al., would enhance the base device of Rao et al. by allowing for it to be implemented accurately and efficiently at high computational speed on computer architecture. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that a computing device would be utilized to perform the operations of the base device of Rao et al. so as to ensure that its operations are carried out accurately, efficiently and at high computational speed on computer architecture. Therefore, it would have been obvious to combine Rao et al. with Houlsby et al. to obtain the invention as specified in claim 17.
- With regards to claim 18, Rao et al. in view of Houlsby et al. disclose the method of claim 17, further comprising applying a model to the frames of the content item, (Rao et al., Pg. 4 § 4 - § 4.1) and providing output from the model to a gated state space model. (Rao et al., Pg. 4 § 4.2 - § 4.3, Pg. 5 Fig. 3 [“a Bi-LSTM”]) Rao et al. fail to disclose explicitly a self-attention model. Pertaining to analogous art, Houlsby et al. disclose applying a self-attention model to the frames of the content item, (Houlsby et al., Abstract, Pg. 2 ¶ 0022 - 0025, Pg. 4 ¶ 0054 - 008, Pg. 5 ¶ 0066 - Pg. 6 ¶ 0072, Pg. 6 ¶ 0078 - Pg. 7 ¶ 0083, Pg. 8 ¶ 0097 - 0099) and providing output from the self-attention model to a model. (Houlsby et al., Figs. 1 & 4, Pg. 4 ¶ 0056 - 0059, Pg. 5 ¶ 0060 - 0063, Pg. 7 ¶ 0090, Pg. 8 ¶ 0105 - 0106) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Rao et al. in view of Houlsby et al. with additional teachings of Houlsby et al. This modification would have been prompted in order to substitute the model of Rao et al. for the self-attention model(s) of Houlsby et al. The self-attention model(s) of Houlsby et al. could be substituted in place of the model of Rao et al. utilizing well-known techniques in the art and would likely yield predictable results, in that in the combination the self-attention model(s) of Houlsby et al. would be utilized to generate the representation(s) characterizing the frame(s) of the content item. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the self-attention model(s) of Houlsby et al. would be utilized to generate the representation(s) characterizing the frame(s) of the content item. Therefore, it would have been obvious to combine Rao et al. in view of Houlsby et al. with additional teachings of Houlsby et al. to obtain the invention as specified in claim 18.
- With regards to claim 19, Rao et al. in view of Houlsby et al. disclose the method of claim 17, further comprising applying a plurality of different models to the frames of the content item, (Rao et al., Pg. 4 § 4 - § 4.1, Pg. 5 Fig. 3) wherein the plurality of different models are configured to focus on different types of visual features; (Rao et al., Pg. 4 § 4 - § 4.1, Pg. 5 Fig. 3) and wherein each of the plurality of different models is configured to provide output to a corresponding gated state space model. (Rao et al., Pg. 4 § 4.2 - § 4.3, Pg. 5 Fig. 3 [“a Bi-LSTM”]) Rao et al. fail to disclose explicitly different self-attention models. Pertaining to analogous art, Houlsby et al. disclose applying a plurality of different self-attention models to the frames of the content item, (Houlsby et al., Abstract, Pg. 2 ¶ 0022 - 0025, Pg. 4 ¶ 0054 - 008, Pg. 5 ¶ 0066 - Pg. 6 ¶ 0072, Pg. 6 ¶ 0078 - Pg. 7 ¶ 0083, Pg. 8 ¶ 0097 - 0099) wherein the plurality of different self-attention models are configured to focus on different types of visual features; (Houlsby et al., Abstract, Pg. 2 ¶ 0022 - 0025, Pg. 4 ¶ 0054 - 008, Pg. 5 ¶ 0066 - Pg. 6 ¶ 0072, Pg. 6 ¶ 0078 - Pg. 7 ¶ 0083, Pg. 8 ¶ 0097 - 0099) and wherein each of the plurality of different self-attention models is configured to provide output to a corresponding model. (Houlsby et al., Figs. 1 & 4, Pg. 4 ¶ 0056 - 0059, Pg. 5 ¶ 0060 - 0063, Pg. 6 ¶ 0079, Pg. 7 ¶ 0090, Pg. 8 ¶ 0105 - 0106) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Rao et al. in view of Houlsby et al. with additional teachings of Houlsby et al. This modification would have been prompted in order to substitute the plurality of different models of Rao et al. for the plurality of different self-attention models of Houlsby et al. The plurality of different self-attention models of Houlsby et al. could be substituted in place of the plurality of different models of Rao et al. utilizing well-known techniques in the art and would likely yield predictable results, in that in the combination the plurality of different self-attention models of Houlsby et al. would be utilized to generate the representation(s) characterizing the frame(s) of the content item. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the plurality of different self-attention models of Houlsby et al. would be utilized to generate the representation(s) characterizing the frame(s) of the content item. Therefore, it would have been obvious to combine Rao et al. in view of Houlsby et al. with additional teachings of Houlsby et al. to obtain the invention as specified in claim 19.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Rao et al., “A Local-to-Global Approach to Multi-modal Movie Scene Segmentation”, arXiv, arXiv:2004.02678v3, 28 Apr. 2020, pages 1 - 10, herein referred to as “Rao et al.”, in view of Houlsby et a l. U.S. Publication No. 2022/0108478 A1 as applied to claim 17 above, and further in view of Chu et al. U.S. Publication No. 2019/0147105 A1.
- With regards to claim 20, Rao et al. in view of Houlsby et al. disclose the method of claim 17, further comprising executing the prediction model to use the scene boundaries to generate segments of the content item. (Rao et al., Pg. 1 Abstract, Pg. 2 Left-Hand Column First-Full Paragraph - Third-Full Paragraph, Pg. 2 Right-Hand Column Third-Full Paragraph, Pgs. 4 - 5 § 4.3 - § 4.4, Pg. 5 Fig. 3, Pg. 8 § 6) Rao et al. fail to disclose explicitly the computing device and controlling playback of the content item based on the segments. Pertaining to analogous art, Chu et al. disclose executing, by the computing device, (Chu et al., Abstract, Fig. 1A, Pg. 1 ¶ 0022 - 0023, Pg. 2 ¶ 0038 - Pg. 3 ¶ 0040, Pg. 3 ¶ 0048, Pg. 4 ¶ 0056, Pg. 5 ¶ 0068 - Pg. 6 ¶ 0075) the prediction model to use the scene boundaries to generate segments of the content item, (Chu et al., Abstract, Figs. 1A - 2, Pg. 1 ¶ 0005 - 0006 and 0020 - 0021, Pg. 2 ¶ 0024 - 0026, 0035 - 0036 and 0039, Pg. 3 ¶ 0044 - 0047, Pg. 4 ¶ 0052 - 0055, Pg. 5 ¶ 0065 - 0067) and to control playback of the content item based on the segments. (Chu et al., Pg. 2 ¶ 0024) Rao et al. in view of Houlsby et al. and Chu et al. are combinable because they are all directed towards image and/or video processing methods that generate one or more representations that characterize an image or video and, similar to Rao et al., Chu et al. is also directed towards video scene segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Rao et al. in view of Houlsby et al. with the teachings of Chu et al. This modification would have been prompted in order to enhance the combined base device of Rao et al. in view of Houlsby et al. with the well-known and applicable technique Chu et al. applied to a comparable device. Controlling playback of the content item based on the segments, as taught by Chu et al., would enhance the combined base device by allowing for it to utilize the scene boundaries it determined in a number of related video processing functions and/or applications, such as smart fast-forwarding applications, so as to expand the number and variety of applications in which the combined base device may be utilized and increase its overall appeal and usefulness to potential end-users. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that playback of the content item would be controlled based on the segments so as to expand the number and variety of applications in which the combined base device may be utilized and increase its overall appeal and usefulness to potential end-users. Therefore, it would have been obvious to combine Rao et al. in view of Houlsby et al. with Chu et al. to obtain the invention as specified in claim 20.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
a. Graves et al., “Framewise phoneme classification with bidirectional LSTM and other neural network architectures”, Neural Networks, Vol. 18, 2005, pages 602 - 610; which is directed towards a description of bidirectional Long Short Term Memory (LSTM) networks.
b. “Long short-term memory”, Wikipedia The Free Encyclopedia, en.wikipedia.org/w/index.php?title=Long_short-term_memory&oldid=1141749728, Feb. 2023, pages 1 - 15; which is directed towards a description of a long short-term memory (LSTM) neural network.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC RUSH whose telephone number is (571) 270-3017. The examiner can normally be reached 9am - 5pm Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270 - 5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERIC RUSH/Primary Examiner, Art Unit 2677
Prosecution Insights