DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/08/2023 is being considered by the examiner.
Specification Objections
The disclosure is objected to because of the following informalities:
In paragraph [0042], line 7, “the video track the corresponding dubbed language audio track” should read “the video track and the corresponding dubbed language audio track.”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 1, 3, 4, 6, 8, 10, and 20, recites limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claim 1; recites the limitation, “…applying the machine-learning model…” [Lines 7, 12].
Claim 1; recites the limitation, “…training a machine-learning model…” [Line 1].
Claim 3; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claim 4; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claim 6; recites the limitation, “…wherein the machine-learning model…” [Line 2].
Claim 8; recites the limitation, “…wherein the machine-learning model…” [Line 2].
Claim 10; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claim 20; recites the limitation “…cause the computing device to:…” [Line 3].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification, the following limitations in claims 1, 3, 4, 6, 8, 10, and 20:
“machine-learning model” (Fig. 3, #300. Paragraph [0007]- the machine-learning model includes convolutional neural network encoders and transformer models that are specialized for processing videos and audio spectrograms (Wherein the machine-learning model is being interpreted as a convolutional neural network encoder and a transformer model, but a transformer model is not known. Thus, the machine-learning model does not have a sufficient structure associated with it).
“computing device” (Fig. 4, #402. Paragraph [0035 and 0052]- computing device is described as computing devices such as the server(s) 402. As shown in FIG. 4, the server(s) 402 includes one or more physical processors, such as the physical processor 418. The physical processor 418 generally represents any type or form of hardware-implemented processing unit capable of interpreting and/or executing computer-readable instructions. In one implementation, the physical processor 418 accesses and/or modifies one or more of the components of the language-invariant audiovisual system 200. Examples of physical processors include,
without limitation, microprocessors, microcontrollers, Graphics Processing Units (GPUs), Central Processing Units (CPUs), Field-Programmable Gate Arrays (FPGAs) that implement softcore processors, Application-Specific Integrated Circuits (ASICs), portions of one or more of the same, variations or combinations of one or more of the same, and/or any other suitable physical processor. (Thus, the computing device have a sufficient structure associated with it processors.).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 3, 4, 6, 8, and 10, along with their dependent claims 2, 5, 7, 9, are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. As described above, the disclosure does not provide adequate structure to perform the claimed function in the recited limitations.
Claims 1, 3, 4, 6, 8, and 10, recite limitations:
Claim 1; recites the limitation, “…applying the machine-learning model…” [Lines 7, 12].
Claim 1; recites the limitation, “…training a machine-learning model…” [Line 1].
Claim 3; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claim 4; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claim 6; recites the limitation, “…wherein the machine-learning model…” [Line 2].
Claim 8; recites the limitation, “…wherein the machine-learning model…” [Line 2].
Claim 10; recites the limitation, “…applying the machine-learning model…” [Line 2].
The specification does not demonstrate that applicant has made an invention that achieves the claimed function because the invention is not described with sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor had possession of the claimed invention.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 3, 4, 6, 8, and 10, along with their dependent claims 2, 5, 7, 9, are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 3, 4, 6, 8, and 10 recite limitations:
Claim 1; recites the limitation, “…applying the machine-learning model…” [Lines 7, 12].
Claim 1; recites the limitation, “…training a machine-learning model…” [Line 1].
Claim 3; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claim 4; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claim 6; recites the limitation, “…wherein the machine-learning model…” [Line 2].
Claim 8; recites the limitation, “…wherein the machine-learning model…” [Line 2].
Claim 10; recites the limitation, “…applying the machine-learning model…” [Line 2].
Claims 1, 3, 4, 6, 8, and 10 invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification is devoid of adequate structure to perform the claimed functions. The specification does not provide sufficient details such that one of ordinary skill in the art would understand which structure performed(s) the claimed function.
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because:
Regarding independent claim 1 and its dependent claims 2-10, claim 1 is directed to a process, which falls within one of the four statutory categories.
Claim 1 recites, in part: “applying… to generate: a first video representation of the video track clip paired with the primary language audio track, and a second video representation of the video track clip paired with the dubbed language audio track; and continually applying… to the training set until the first video representation and the second video representation are positioned within a threshold distance from each other within a representational space.”
The limitations, as drafted above, are processes that, under broadest reasonable interpretation (BRI), cover the performance of the limitation in the mind, which falls within the “Mental Processes” grouping of abstract ideas. The limitation of “applying… to generate: a first video representation of the video track clip paired with the primary language audio track, and a second video representation of the video track clip paired with the dubbed language audio track” is a step, under BRI, that a human can also perform through mental processes such as observation and evaluation. For instance, a human mind can observe two different videos and two different audio tracks and evaluate them to pair them together according to certain given conditions.
The limitation of “and continually applying… to the training set until the first video representation and the second video representation are positioned within a threshold distance from each other within a representational space” is a step, under BRI, that a human can also perform through mental processes of observation and evaluation; for example, the human mind can observe a training process on some training set until a certain result is met (the result here being the two videos being positioned within a threshold distance).
Accordingly, the claim recites an abstract idea. This judicial exception is not integrated into a practical application. In particular, claim 1 recites the following additional elements:
“training a machine-learning model to generate language-invariant video representations generating a training set comprising, from a long-form video, a video track clip, a primary language audio track corresponding to the video track clip, and a dubbed language audio track corresponding to the video track clip; the machine-learning model”
The additional elements “training… video representations” is part of the preamble indicating an intended use; “generating… video track clip” is an insignificant extra solution activity of data gathering; “the machine-learning model” is a generic, well-known neural network model recited at a high level of generality, it is in the claim as a mere attempt to implement the abstract ideas/judicial exceptions using a generic neural network model without further limiting how, in detail, the model works to arrive at such an outcome.
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea. Please see MPEP §2106.04.(d).III.C.
There are no additional elements, such as for these additional elements as indicated above, that amount to significantly more than the judicial exception. Please see MPEP §2106.05. The claim is directed to an abstract idea.
For all of the foregoing reasons, claim 1 does not comply with the requirements of 35 USC 101. Accordingly, the dependent claims 2-9 do not provide elements that overcome the deficiencies of the independent claim 1.
Moreover, claims 2, 5 and 10 recite, in part, clauses of merely further specification of the element which each of them depends on, and therefore are not an indication of an integration of the abstract ideas into a practical application, nor are they considered significantly more.
Claim 3 recites, in part, “applying….to the training set to generate: video representations of the additional video track clips paired with the primary language audio tracks corresponding to the additional video track clips, and video representations of the additional video track clips paired with the dubbed language audio tracks corresponding to the additional video track clips.” This is a step, under BRI, that a human can also perform through mental processes of observation and evaluation. For instance, a human mind can observe two different videos and two different audio tracks and evaluate them to pair them together according to a certain condition already given; “the machine-learning model” is a generic well-known neural network model recited at high level of generality, it is in the claim as a mere attempt to implement the abstract ideas/judicial exceptions using a generic neural network model without further limiting how, in detail, the model works to arrive at such an outcome.
Claim 4 recites, in part, “continually applying….to the training set until the video representations of the additional video track clips paired with the primary language audio tracks corresponding to the additional video track clips and the video representations of the additional video track clips paired with the dubbed language audio tracks corresponding to the additional video track clips are positioned within the threshold distance from each other within the representational space.” This is a step, under BRI, that a human can also perform through mental processes of observation and evaluation. For example, the human mind can observe a training process on some training set until a certain result is met (the result here being the two videos are positioned within a certain threshold distance from one another); “the machine-learning model” is a generic well-known neural network model recited at a high level of generality, it is in the claim as a mere attempt to implement the abstract ideas/judicial exceptions using a generic neural network model without further limiting how, in detail, the model works to arrive at such an outcome.
Claim 6 recites, in part, “wherein the machine-learning model comprises convolutional neural network encoders and transformer models that are specialized for processing videos and audio spectrograms.” The claim includes a wherein clause of giving further specification on what the additional element “machine-learning model” comprises of, without further limiting how, in detail, the model works to arrive at such outcome; here, the model comprises of generic machine learning model components recited at high level of generality such as CNN encoders, transformer performing generic functions of processing data/information (here being videos and audio spectrograms).
Claims 7-9, in part, wherein clauses of merely further specification of the element that each of them depends on, and therefore are not indications of an integration of the abstract ideas into a practical application nor, considered significantly more. Moreover, claim 8 recites further features of terms such as, “multi-layer perception heads,” which is a well-known generic term in the field of neural network, recited at high level of generality to perform a generic function of “projecting….the representation….”.
Accordingly, the dependent claims 2-10 are not patent eligible under 101.
Regarding independent claim 11 and its dependent claims 12-19:
The independent claim 11 recites analogous limitations to the independent claim 1 hence, these analogous limitations are not 101 eligible for the reasons above in the claim 1 analysis. Furthermore, claim 11 recites some additional features such as, “a system comprising: at least one physical processor; and physical memory comprising computer-executable instructions that, when executed by the at least one physical processor, cause the at least one physical processor to perform acts,” which are features of generic computers and computer components recited at a high level of generality to perform generic well-known functions such as a processor processing instructions stored in a memory, etc.
The dependent claims 12-19 each recite analogous limitations to the dependent claims 2-10, hence, these analogous limitations are not 101 eligible for the reasons above in the analysis above.
Regarding independent claim 20:
The independent claim 20 recites analogous limitations to the independent claim 1 hence, these analogous limitations are not eligible for the reasons above in the claim 1 analysis. Furthermore, claim 20 recites some additional features such as, “a non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to…” which are features of generic computers and computer components recited at a high level of generality to perform generic well-known functions such as a processor processing instructions stored in a memory, etc. The claim further recites, “generate a training set comprising… a second dubbed language audio track corresponding to the video track clip…” which merely adds further specification to the analogous limitation of independent claim 1, wherein the additional element introduced is an insignificant extra-solution activity of data gathering and thus does not overcome the identified deficiencies. Additionally, the claim recites, “apply… to the training set to generate… a third video representation of the video track clip paired with the second dubbed language audio track” which is a step, under BRI, that a human can also perform through mental processes such as observation and evaluation, for instance, a human mind can observe three different videos and three different audio tracks and evaluate them to group them together according to a certain condition already given. The claim also recites, “…and continually apply the machine-learning model to the training set until the first video representation, the second video representation, and the third video representation are positioned within a threshold distance from each other within a representational space” which is a step, under BRI, a human can also perform through mental processes of observation and evaluation such as, the human mind can observe a training process on some training set until a certain result is met, the result here being the three videos are within a threshold distance. Accordingly, the claim recites an abstract idea and se additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea. Please see MPEP §2106.04.(d).III.C.
Conclusion
Listed below are the prior arts made of record and not relied upon is considered pertinent to applicant's disclosure.
Fei et al. (US 20210390270 A1) - Presented herein are embodiments of an unsupervised cross-lingual sentiment classification model (which may be referred to as multi-view encoder-classifier (MVEC)) that leverages an unsupervised machine translation (UMT) system and a language discriminator. Unlike previous language model (LM)-based fine-tuning approaches that adjust parameters solely based on the classification error on training data, embodiments employ an encoder-decoder framework of an UMT as a regularization component on the shared network parameters. In one or more embodiments, the cross-lingual encoder of embodiments learns a shared representation, which is effective for both reconstructing input sentences of two languages and generating more representative views from the input for classification. Experiments on five language pairs verify that an MVEC embodiment significantly outperforms other models for 8/11 sentiment classification tasks.…….. Figs. 1, 2. Abstract.
Wang et al. (US 11983923 B1)- The disclosed computer-implemented method may include receiving, as input, an audio/video data object; isolating a video stream of a visible potential speaker over a plurality of frames of the audio/video data object; isolating an audio stream over the plurality of frames; providing the isolated video stream and the isolated audio stream to a machine learning model trained with contrastive learning, the contrastive learning using (i) a corpus of video segments of visible speakers with corresponding original audio for positive samples; and (ii) a corpus of video segments of visible speakers with corresponding dubbed audio for negative samples; and evaluating a match between the isolated audio stream and the isolated video stream based at least in part on an output of the machine learning model. Various other methods, systems, and computer-readable media are also disclosed.…. Figs. 1, 2. Abstract.
Fei et al. (US 20220383048 A1)- Current pretrained vision-language models for cross-modal retrieval tasks in English depend upon on the availability of many annotated image-caption datasets for pretraining to have English text. However, the texts are not necessarily in English. Although machine translation (MT) tools may be used to translate text to English, the performance largely relies on MT's quality and may suffer from high latency problems in real-world applications. Embodiments herein address these problems by learning cross-lingual cross-modal representations for matching images and their relevant captions in multiple languages. Embodiments seamlessly combine cross-lingual pretraining objectives and cross-modal pretraining objectives in a unified framework to learn image and text in a joint embedding space from available English image-caption data, monolingual corpus, and parallel corpus. Embodiments are shown to achieve state-of-the-art performance in retrieval tasks on multimodal multilingual image caption datasets.….. Fig. 4, Abstract.
Thomson et al. (US 20250086408 A1)- A method may include obtaining a first video that includes sign language content. In some embodiments, the sign language content may include one or more video frames of a figure performing sign language. The method may also include obtaining language data that represents the sign language content in the first video and creating a second video including sign language content by altering the first video. The method may further include training a machine learning model of a translation system configured to translate between sign language and language data using the second video and the language data…. Fig. 2A, 2B, 3, Abstract.
Puri et al. (US 20230394250 A1)- Present disclosure generally relates to machine translation systems, and particularly to method and system for cross-lingual adaptation using disentangled syntax and shared conceptual latent space for low-resource natural languages. Method includes converting multi-lingual sentences received from user, to linearized constituency parse tree and mask leaf nodes in linearized constituency parse tree to separate semantic information in multi-lingual sentences. Method includes passing linearized constituency parse tree with masked leaf nodes, to syntactic encoder for disentangling syntactic information in multi-lingual sentences. Method includes determining, from syntactic information, if multi-lingual sentences include new language to be learned which includes new script relatively to pre-existing language in language model and unique script with similarities in sentence structure corresponding to pre-existing language.…. Abstract.
Concolato (US 20250324117 A1)- In various embodiments, a caption encoding application performs captioning while streaming live events. The caption encoding application determines a segment index in response to a triggering event associated with a live event. The caption encoding application computes a caption time interval based on the first triggering event and a caption delay. The caption encoding application retrieves from a database a portion of caption data based on the caption time interval and a language. The caption encoding application generates a caption segment based on the portion of the caption data. The caption encoding application causes the caption segment to be inserted into a caption stream at the segment index, where the caption stream is to be transmitted as part of the live event..…. Abstract.
Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." Proceedings of the IEEE international conference on computer vision. 2017 - We consider the question: what can be learnt by looking at and listening to a large number of unlabeled videos? There is a valuable, but so far untapped, source of information contained in the video itself – the correspondence between the visual and the audio streams, and we introduce a novel “Audio-Visual Correspondence” learning task that makes use of this. Training visual and audio networks from scratch, without any additional supervision other than the raw unconstrained videos themselves, is shown to successfully solve this task, and, more interestingly, result in good visual and audio representations…. Abstract.
Patrick, Mandela, et al. "Support-set bottlenecks for video-text representation learning." arXiv preprint arXiv:2010.02824 (2020)- The dominant paradigm for learning video-text representations -- noise contrastive learning -- increases the similarity of the representations of pairs of samples that are known to be related, such as text and video from the same sample, and pushes away the representations of all other pairs. We posit that this last behaviour is too strict, enforcing dissimilar representations even for samples that are semantically-related -- for example, visually similar videos or ones that share the same depicted action. In this paper, we propose a novel method that alleviates this by leveraging a generative model to naturally push these related samples together: each sample's caption must be reconstructed as a weighted combination of other support samples' visual representations… Abstract.
Morgado, Pedro, Yi Li, and Nuno Nvasconcelos. "Learning representations from audio-visual spatial alignment." Advances in Neural Information Processing Systems 33 (2020): 4733-4744- We introduce a novel self-supervised pretext task for learning representations from audio-visual content. Prior work on audio-visual representation learning leverages correspondences at the video level. Approaches based on audio-visual correspondence (AVC) predict whether audio and video clips originate from the same or different video instances. Audio-visual temporal synchronization (AVTS) further discriminates negative pairs originated from the same video instance but at different moments in time. While these approaches learn high-quality representations for downstream tasks such as action recognition, they completely disregard the spatial cues of audio and visual signals naturally occurring in the real world. To learn from these spatial cues, we tasked a network to perform contrastive audio-visual spatial alignment of 360\degree video and spatial audio… Abstract.
Zhao, Han, Junjie Hu, and Andrej Risteski. "On learning language-invariant representations for universal machine translation." International conference on machine learning. PMLR, 2020- The goal of universal machine translation is to learn to translate between any pair of languages. Despite impressive empirical results and an increasing interest in massively multilingual models, theoretical analysis on translation errors made by such universal machine translation models is only nascent. In this paper, we formally prove certain impossibilities of this endeavour in general, as well as prove positive results in the presence of additional (but natural) structure of data. For the former, we derive a lower bound on the translation error in the many-to-many translation setting, which shows that any algorithm aiming to learn shared sentence representations among multiple language pairs has to make a large translation error on at least one of the translation tasks, if no assumption on the structure of the languages is made. For the latter, we show that if the paired documents in the corpus follow a natural\emph {encoder-decoder} generative process, we can expect a natural notion of “generalization”: a linear number of language pairs, rather than quadratic, suffices to learn a good representation… Abstract.
Spiteri Miggiani, Giselle. "Measuring quality in translation for dubbing: a quality assessment model proposal for trainers and stakeholders." (2022)- Quality assessment in the field of Audiovisual Translation (AVT) has been addressed by several scholars, particularly in relation to interlingual subtitling (Pedersen, 2017; Robert & Remael, 2016), intralingual live subtitling (Romero-Fresco & Martínez Pérez, 2015) and interlingual live subtitling (Robert & Remael, 2017; Romero-Fresco & Pöchhacker, 2017), but to-date no model in relation to dubbing has been proposed. As with other AVT modes, the need for a quality assessment method in dubbing arises in academic and in-house training contexts. Moreover, localization companies often resort to ‘entry tests’ before engaging translators. Self-assessment also proves to be one of the main challenges for trainees in a dubbing training context, and any quality assessment tools can possibly be of help.. .Abstract.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEZAWIT N SHIMELES whose telephone number is (571)272-7663. The examiner can normally be reached M-F 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEZAWIT NOLAWI SHIMELES/ Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/ Supervisory Patent Examiner, Art Unit 2673