DETAILED ACTION
Introduction
Claims 1-7 and 12 have been examined in this application. Claims 1-7 and 12 are amended. Claims 8-11 and 13 are hereby withdrawn (see Election/Restrictions below). Claims 14 and 15 are cancelled.
This is the First Action On the Merits (FAOM), in response to Applicant’s response (filed 2/27/2026) to the Election / Restriction requirement (mailed 12/31/2025). The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Office Action Formatting
The following is an explanation of the formatting used in the instant Office Action:
• [0001] – Indicates a paragraph number in the most recent, previously cited source;
• [0001, 0010] – Indicates multiple paragraphs (in example: paragraphs 1 and 10) in the most recent, previously cited source;
• [0001-0010] – Indicates a range of paragraphs (in example: paragraphs 1 through 10) in the most recent, previously cited source;
• 1:1 – Indicates a column number and a line number (in example: column 1, line 1) in the most recent, previously cited source;
• 1:1, 2:1 – Indicates multiple column and line numbers (in example, column 1, line 1 and column 2, line 2) in the most recent, previously cited source;
• 1:1-10 – Indicates a range of lines within one column (in example: all lines spanning, and including, lines 1 and 10 in column 1) in the most recent, previously cited source;
• 1:1-2:1 – Indicates a range of lines spanning several columns (in example: column 1, line 1 to column 2, line 1 and including all intervening lines) in the most recent, previously cited source;
• p. 1, ln. 1 – Indicates a page and line number in the most recent, previously cited source;
• ¶1 – The paragraph symbol is used solely to refer to Applicant's own specification (further example: p. 1, ¶1 indicates first paragraph of page 1); and
• BRI – the broadest reasonable interpretation.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on application EP23197545.9 filed in Europe on 09/14/2023. It is noted, however, that applicant has not filed a certified copy of the application as required by 37 CFR 1.55 (see Failure Status Report regarding the failure to retrieve the priority document, mailed 2/14/2025).
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9/23/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the IDS is being considered by the examiner.
Election/Restrictions
Applicant's election with traverse of Invention I, in the reply filed on 2/27/2026 is acknowledged. The traversal is on the ground(s) that undue burden would not be presented, as Inventions I and II are sufficiently related. This is not found persuasive because although the inventions are related as subcombination and combination, as detailed in the Requirement for Restriction/Election, the two inventions require different limitations such as the retrieving, storing, analyzing deciding, and triggering of Invention I which would require a different search strategy when compared with the exchanging over the air, querying/retrieving with specific time parameters, analyzing at a global level, and deciding, in the combination claim. Additionally, the combination does not require all of the particular of the subcombination claims. The requirement is still deemed proper and is therefore made FINAL. Claims 8-11 and 13 are therefore withdrawn.
Claim Objections
Claim 1 is objected to because of the following informalities:
In Claim 1, in the phrase “analyzing, with the computing unit, analyzing the sensor data,” one of the words “analyzing” should be removed.
In Claim 1, “in case” should instead read “in a case when” or similar.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Such claim limitations are:
(a) a “data logging unit” performing the retrieving of sensor data and storing functions, in Claim 1,
(b) a “computing unit” performing the retrieving and analyzing functions, in Claim 1,
(c) a “relevancy determination unit” performing the retrieving and storing, deciding, and triggering functions, in Claim 1,
because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. A review of the specification shows that the following appears to be the corresponding structure described in the specification for the 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph limitation:
(a), (b), (c): the specification is unclear regarding the corresponding structure of limitations (a) through (c), see 112(b).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
This application includes one or more claim limitations that use the word “means” or “step” but are nonetheless not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph because the claim limitation(s) recite(s) sufficient structure, materials, or acts to entirely perform the recited function. Such claim limitations are:
(d) “wherein the method comprises the steps of” in Claim 1,
(d) “f) continue with step a)” in Claim 7.
Examiner’s note: “If the claim element uses the phrase ‘step for,’ then Section 112, Para. 6 is presumed to apply…. On the other hand, the term ‘step’ alone and the phrase ‘steps of’ tend to show that Section 112, Para. 6 does not govern that limitation." See MPEP 2181 (I) (A).
Because this/these claim limitation(s) is/are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are not being interpreted to cover only the corresponding structure, material, or acts described in the specification as performing the claimed function, and equivalents thereof.
If applicant intends to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to remove the structure, materials, or acts that performs the claimed function; or (2) present a sufficient showing that the claim limitation(s) does/do not recite sufficient structure, materials, or acts to perform the claimed function.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-7 and 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 1, claim limitations (a) “data logging unit, ”(b) “computing unit,” and (c) “relevancy determination unit” invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. Particularly, the specification recites the limitations of the data logging unit, computing unit, and relevancy determination unit by name, but does not appear to recite any particular structure that corresponds to these limitations. It is not clear whether the units correspond to sensors, or computer processors and algorithms, or remote servers or communication circuitry, or some combination of these things. Specification ¶0028 recites that the computing unit comprises data encoders, but in ¶0029 these appear to be recited only as software and not any particular structure. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. For the purposes of examination, the units are interpreted as processor(s) and algorithms for performing the functions.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Additionally, regarding Claim 1, the limitation of deciding whether the perception results or neural network embeddings are considered “relevant or irrelevant” renders the claim indefinite. The term “relevant” is generally defined as ”relating in an appropriate way to something under consideration.” However, the claim does not establish what is “under consideration” for the results or embeddings to be considered relevant in relation to. It is not clear whether the deciding is about whether the results or embeddings are relevant for vehicle control, or relevant to an insurance claim, or relevant to safety, or efficiency, or something else entirely. The scope of the claim is therefore indefinite. For the purposes of examination, the limitation is interpreted as determining whether the perception results or neural network embeddings are candidates for labelling or not.
Claims 2-7 and 12 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as being dependent on rejected Claim 1 and for failing to cure the deficiencies listed above.
Regarding Claim 2, the claim recites determination of relevancy of perception results or the neural network embeddings by applying a predefined rule set to the retrieved perception results (and not to the neural network embeddings). However, Claim 1 recites the perception results and neural network embeddings in the alternative (i.e. “at least one”), such that only one is required under the broadest reasonable interpretation of the claim. For the case of Claim 2 where the determination of relevancy is of the neural network embeddings, by applying the rule set to the retrieved perception results, it is not clear whether this is stating that both of the perception results and neural network embeddings are now required in the claim, or alternatively whether the claim is only intended to encompass the determination of relevancy of the perception results, or whether the determination is optional/not required for the case of only neural network embeddings being present in Claim 1. The scope of the claim is therefore indefinite. For the purposes of examination, the claim is interpreted as requiring at least the perception results but not necessarily the neural network embeddings.
Regarding Claims 3, 5 and 7, the claims further narrow features corresponding to the neural network embeddings. However, Claim 1 recites the perception results and neural network embeddings in the alternative (i.e. “at least one”), such that only one is required under the broadest reasonable interpretation of the claim. It is not clear, for the case when the BRI of Claim 1 pertains to the perception results, whether the limitations in Claims 3 and 5 are optional/not required under the BRI, or alternatively whether Claims 3 and 5 should be interpreted such that the at least one of the perception results and neural network embeddings now must include the neural network embeddings. The scope of the claim is therefore indefinite. For the purposes of examination, the claims are interpreted as requiring the neural network embeddings, i.e. narrowing “perception results or neural network embeddings” to now being neural network embeddings and optionally also the perception results.
Regarding Claim 4, the claim further narrow features corresponding to the perception results. However, Claim 1 recites the perception results and neural network embeddings in the alternative (i.e. “at least one”), such that only one is required under the broadest reasonable interpretation of the claim. It is not clear, for the case when the BRI of Claim 1 pertains to the neural network embeddings, whether the limitations in Claim 4 are optional/not required under the BRI, or alternatively whether Claim 4 should be interpreted such that the at least one of the perception results and neural network embeddings now must include the perception results. The scope of the claim is therefore indefinite. For the purposes of examination, the claim is interpreted as now requiring the perception results.
Regarding Claim 6, the claim recites determination of relevancy of perception results or the neural network embeddings based on a comparison using the neural network embeddings (and not the perception results). However, Claim 1 recites the perception results and neural network embeddings in the alternative (i.e. “at least one”), such that only one is required under the broadest reasonable interpretation of the claim. For the case of Claim 6 where the perception results are evaluated based on the neural network embeddings, it is not clear whether this is stating that both of the perception results and neural network embeddings are now required in the claim, or alternatively whether the claim is only intended to encompass the determination of relevancy of the neural network embeddings, or whether the determination is optional/not required for the case of only perception results being present in Claim 1. The scope of the claim is therefore indefinite. For the purposes of examination, the claim is interpreted as determination of the relevance of neural network embeddings based on the comparison, and the claim requiring the neural network embeddings.
Claim 7 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being dependent on rejected Claim 6 and for failing to cure the deficiencies listed above.
Regarding Claim 7, the phrase “the whole dataset of neural network embeddings” renders the claim indefinite. It is not clear whether this is the same as the “predetermined number of neural network embeddings” which was previously recited, or alternatively whether the “whole” set is some larger set from which the “predetermined number” is selected. Additionally, Regarding Claim 7, the limitation of “a running mean of at least one of the determined distance or the density measurements for the smaller subsets of the predetermined number of neural network embeddings” renders the claim indefinite. There is no antecedent basis for “the smaller subsets.” It is not clear whether this is the same as the predetermined number of neural network embeddings (i.e. smaller compared to the whole) or is referring to the temporary coherent subsets, or something else. The scope of the claim is therefore indefinite. For the purposes of examination, the phrase and claim as a whole are interpreted as any evaluation of k nearest neighbors distance to determine relevance.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 6, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Publication US2022/0121877A1 (Maoz et al.) in view of Publication US2019/0302766A1 (Mondello et al.).
Regarding Claim 1, Maoz et al. discloses a method of operating a data recording device (see Figure 3, method for recording/evaluating sensor data, and Figures 4, 5, device) for applications for vehicles (see [0041] annotation of vehicle data), the data recording device comprising:
a data logging unit, a computing unit (see [0087-0089] processor and instructions), a storage unit (see [0087] memory and/or non-transitory data storage) and a relevancy determination unit including at least one relevancy determination module (see [0087-0089]),
wherein the method comprises the steps of:
retrieving, with the data logging unit and the computing unit, sensor data acquired by in-vehicle sensors (see [0066] step 302, sensor data sets determined, [0041] for sensors of a vehicle),
storing, with the data logging unit, the sensor data (see [0087-0089] carried out using data in memory),
analyzing, with the computing unit, the sensor data and using the sensor data to derive at least one of perception results (see [0066] step 304 determine signature which [0045] can be a vector representing a scene, or alternatively [0049] a neural network embedding as the perception result) or neural network embeddings from the retrieved sensor data (see [0066] step 304 determine signature which [0049] may be a neural network embedding),
retrieving and storing, with the relevancy determination unit, the at least one of the perception results or the neural network embeddings (see [0066] step 306 signatures (results/embeddings) used in step 306, [0087-0089] implemented by program/memory),
deciding, with the relevancy determination unit, whether the retrieved at least one of the perception results or the neural network embeddings are considered relevant or irrelevant (see [0066] step 306 determine whether sensor data set is candidate for labelling [0068] based on evaluation of respective signature),
triggering, with the relevancy determination unit, a providing of the sensor data in case the at least one of the perception results or the neural network embeddings corresponding to the sensor data are considered relevant by the relevancy determination unit (see [0066] step 308 providing to labelling instance if sensor data set is candidate).
Maoz et al. does not explicitly recite the method:
for Autonomous Driving or Advanced Driver Assistant Systems applications for vehicles,
the storage unit including a circular buffer and a persistent storage region,
storing the sensor data in the circular buffer,
triggering, with the relevancy determination unit, a transfer of the sensor data from the circular buffer to the persistent storage region for persistent storage of the sensor data in case the at least one of the perception results or the neural network embeddings corresponding to the sensor data are considered relevant by the relevancy determination unit.
However, Mondello et al. teaches a method for selecting relevant data (see [0009, 0032-0033] analyze sensor stream to determine event of interest is imminent or impending),
for Autonomous Driving or Advanced Driver Assistant Systems applications for vehicles (see [0009, 0011]),
the storage unit including a circular buffer and a persistent storage region (see [0013-0016] sensor data initially held in circular buffer, and [0017] non-volatile storage device 112),
storing the sensor data in the circular buffer (see [0013-0016]),
triggering, with the relevancy determination unit, a transfer of the sensor data from the circular buffer to the persistent storage region for persistent storage of the sensor data (see Figure 3, [0033] step 308 store the data stream of sensor data in the non-volatile storage device 112) in case the at least one of the perception results (see Figure 3, [0032] analyzed sensor data at 304) or the neural network embeddings corresponding to the sensor data are considered relevant by the relevancy determination unit (see Figure 3, for “yes” relevant event).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the analyzing and outputting of sensor data to find candidates for labelling in the method of Maoz et al. to be used to save event data to a persistence storage as taught by Mondello et al., with a reasonable expectation of success, with the motivation of enhancing the selection of data to events of interest and reducing wear and improving longevity by causing less write operations to the non-volatile storage (see Mondello et al., [0010]).
Regarding Claim 2, Maoz et al. discloses the method of claim 1, wherein the deciding, with the at least one relevancy determination module of the relevancy determination unit, whether the retrieved at least one of the perception results or the neural network embeddings are considered relevant or irrelevant includes applying a predefined rule set to the retrieved perception results (see [0050, 0068-0069] similarity evaluation by distance / selecting lowest as set of rules to determine relevant).
Regarding Claim 3, Maoz et al. discloses the method of claim 1, further comprising reducing, with the computing unit, the dimensionality of the neural network embeddings by Principal Component Analysis (see [0049]), transmitting, with the computing unit, the dimensionality reduced neural network embeddings to the relevancy determination unit (see Figure 3, signature to step 306), and deciding, with the relevancy determination unit, on the relevancy based on the dimensionality reduced neural network embeddings (see [0066] step 306 determine whether candidate for labeling).
Regarding Claim 6, Maoz et al. discloses the method of claim 1, wherein the at least one relevancy determination module bases its decision whether the at least one of the retrieved perception results or the neural network embeddings are considered relevant or irrelevant on a comparison of the neural network embeddings with neural network embeddings previously stored in the relevancy determination unit (see [0050] comparison by distance evaluation to “given scene”).
Regarding Claim 12, Maoz et al. discloses a data recording device configured to perform the method of claim 1 (see Figures 4, 5, [0080-0088], hardware performing the method, as modified as specified in the rejection of Claim 1, above).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Publication US2022/0121877A1 (Maoz et al.) in view of Publication US2019/0302766A1 (Mondello et al.), further in view of Publication US2020/0317194A1 (Yan et al.).
Regarding Claim 4, Maoz et al. further discloses determination that perception results are relevant based on user input (see [0053-0054] user can select seed sensor data set by selecting image).
Maoz et al. does not explicitly recite the method of claim 1, wherein the data recording device further comprises a display unit provided with a display and a touch panel, and wherein the method further comprises transmitting, with the computing unit, the perception results to the display unit displaying, with the display unit, the perception results on the display, recognizing, with the display unit, a manual user interaction on the touch panel responsive to the perception results being displayed on the display, and, upon recognizing the manual user interaction, transmitting, with the display unit, the information to the relevancy determination unit that the perception results are relevant.
However, Yan et al. teaches a technique for a selecting particular sensor data, which:
comprises a display unit provided with a display and a touch panel (see [0195] touch-screen), and wherein the method further comprises transmitting, with the computing unit, the perception results to the display unit displaying, with the display unit, the perception results on the display (see [0195] image with plural regions, and [0145] implemented by processor), recognizing, with the display unit, a manual user interaction on the touch panel responsive to the perception results being displayed on the display (see [0195] touching regions that are desired to act as training regions), and, upon recognizing the manual user interaction, transmitting, with the display unit, the information to the relevancy determination unit that the perception results are relevant (see [0195] defining the respective training regions).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the user selection of relevant data of Maoz et al. to be implemented using a touchscreen as taught by Yan et al., with a reasonable expectation of success, with the motivation of allowing for selection of particular portions of data as relevant and improving user convenience by providing an intuitive way to select such data (see Yan et al. [0193-0195]).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Publication US2022/0121877A1 (Maoz et al.) in view of Publication US2019/0302766A1 (Mondello et al.), further in view of Publication US2023/0252795A1 (Tong et al.).
Regarding Claim 5, Maoz et al. further discloses deciding, with the at least one relevancy determination module, whether neural network embeddings derived from sensor data are considered relevant or irrelevant based on a similarity measure, between the neural network embeddings derived from sensor data and user selected embeddings. (see [0053-0054] user can select seed sensor data set by selecting image, [0050] similarity using distance evaluation).
Maoz et al. does not explicitly recite the method of claim 1, further comprising: receiving, with the computing unit, text-based user-queries, encoding, with the computing unit, the retrieved text-based user queries into neural network embeddings, transmitting, with the computing unit, the neural network embeddings to the relevancy determination unit, and deciding, with the at least one relevancy determination module, whether neural network embeddings derived from sensor data are considered relevant or irrelevant based on a similarity measure, using cosine-similarity, between the neural network embeddings derived from sensor data and the encoded user query neural network embeddings.
However, Tong et al. teaches a technique in evaluation of relevant sensor data (see e.g. [0035]),
comprising: receiving, with the computing unit (see [0033] implemented using microprocessor), text-based user-queries (see Claim 1, text encoder [0045-0046] receives concepts in the form of text), encoding, with the computing unit, the retrieved text-based user queries into neural network embeddings (see Claim 1, generate concept embeddings associated with each of a plurality of concepts, [0050] neural network embeddings), transmitting, with the computing unit, the neural network embeddings to the relevancy determination unit, and deciding, with the at least one relevancy determination module, whether neural network embeddings derived from sensor data are considered relevant or irrelevant based on a similarity measure, using cosine-similarity, between the neural network embeddings derived from sensor data and the encoded user query neural network embeddings (see Claim 1, confidence module which generates confidence based on image embeddings (sensor) and concept embeddings, and see Claim 7 by applying cosine similarity function).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the user input of seed data of Maoz et al. to use a text input and encoding as taught by Tong et al., with a reasonable expectation of success, with the motivation of enhancing the robustness and flexibility of the method to allow for additional inputs of seed data and allowing for requesting of relevant data of particular objects (see Tong et al., [0002-0003]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Publication US2022/0121877A1 (Maoz et al.) in view of Publication US2019/0302766A1 (Mondello et al.), further in view of Patent U.S. 10,289,910 B1 (Chen et al.).
Regarding Claim 7, Maoz et al. does not explicitly recite the method of claim 6, wherein the at least one relevancy determination module performs the following:
a) retrieving a predetermined number of neural network embeddings, derived from sensor data taken at consecutive times,
b) determining, for each of the neural network embeddings, nearest neighbors within the whole dataset of neural network embeddings retrieved by the at least one relevancy determination module,
c) at least one of determining, for each of the neural network embeddings, a distance to the nearest neighbors or evaluating a density of the neural network embedding within the whole dataset of neural network embeddings retrieved by the relevancy determination module, by applying a “k-nearest-neighbors” density estimation method,
d) calculating at least one of average distances or average densities for temporary coherent subsets of the predetermined number of neural network embeddings, the at least one of the average distances or the average densities being calculated as a running mean of at least one of the determined distance or the density measurements for the smaller subsets of the predetermined number of neural network embeddings,
e) evaluating the subset having at least one of an average distance larger than a distance threshold or an average density smaller than a density threshold, and considering the respective subset as relevant,
f) continue with step a).
However, Chen et al. teaches a technique to extract relevant sensor data (see Claim 1, object recognition), comprising:
a) retrieving a predetermined number of neural network embeddings, derived from sensor data taken at consecutive times (see 9:30-32, computing CNN features for every 5th video frame, 8:45-50, extractor 300 generating CNN feature vectors 401),
b) determining, for each of the neural network embeddings, nearest neighbors within the whole dataset of neural network embeddings retrieved by the at least one relevancy determination module (see 10:5-10, for each incoming feature from extractor 300, return k nearest neighbors, from dictionary which (see 9:44-47) is made of the dataset),
c) at least one of determining, for each of the neural network embeddings, a distance to the nearest neighbors (see 10:5-20, using distance metric) or evaluating a density of the neural network embedding within the whole dataset of neural network embeddings retrieved by the relevancy determination module, by applying a “k-nearest-neighbors” density estimation method,
d) calculating at least one of average distances (see 10:21-30 average distance to k of the nearest neighbors) or average densities for temporary coherent subsets of the predetermined number of neural network embeddings (see 10:21-30 for each object in feature dictionary, i.e. for all objects including any temporary coherent subset), the at least one of the average distances or the average densities being calculated as a running mean of at least one of the determined distance or the density measurements for the smaller subsets of the predetermined number of neural network embeddings (see 10:21-30 for example for first two objects that are evaluated) ,
e) evaluating the subset having at least one of an average distance larger than a distance threshold or an average density smaller than a density threshold, and considering the respective subset as relevant (see 10:30-64, evaluating set in average having distances (distance being a state of two things apart, i.e. greater than 0 threshold), and probability to determine whether input image contains the object (relevant)),
f) continue with step a) (see 7:25 real-time, i.e. continuous process).
Examiner’s note: see also the rejection under 112(b) and interpretation, above.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the method of Maoz et al. to further determine relevance as taught by Chen et al., with a reasonable expectation of success, with the motivation of improving the versatility of the method to perform the relevance determination on video sensor data in real-time (see Chen et al., 1:35-56).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Paul Allen whose telephone number is (571). The examiner can normally be reached Monday - Friday from 9am to 5pm, Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Piateski can be reached at 571-270-7429. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.A./Examiner, Art Unit 3669
/Erin M Piateski/Supervisory Patent Examiner, Art Unit 3669
Prosecution Insights