DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This action is in response to application filed on 4/25/2024, in which claims 1 – 13 was presented for examination.
3. Claims 1 – 13 are pending in the application.
Information Disclosure Statement
4. The information disclosure statement (IDS) submitted on 5/8/2024 has been reviewed and entered into the record. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1 - 13 are directed are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
As per claim 1,
Step 1: Claim 1 recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The claim recites the limitation of
providing a Topological Data Analysis AI/ML Pipeline (TDAML) algorithm for multimodal sensor data fusion in an autonomy applications system (Mental Process performed in human mind using a pen and paper (i.e. observation)).
combining raw heterogeneous multimodal sensor data at the topological level (Mental Process performed in human mind using a pen and paper (i.e. evaluation)).
measuring, recording, and tracking linear representations of an underlying data set (Mental Process performed in human mind using a pen and paper (i.e. evaluation)).
providing a linear representation of the underlying data set which is compatible to existing deep learning (DL) model architectures for training in autonomy tasks (Mental Process performed in human mind using a pen and paper (i.e. judgement)).
accessing the entire degree of freedom (DOF) space of raw multimodal sensor data for mitigating sensor modality adversarial threats and environmental attenuation concerns in contested military and civilian (urban) environments (Mental Process performed in human mind using a pen and paper (i.e. judgement)).
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application. The claim recites the additional elements of
providing a linear representation of the underlying data set which is compatible to existing deep learning (DL) model architectures for training in autonomy tasks (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g))
accessing the entire degree of freedom (DOF) space of raw multimodal sensor data for mitigating sensor modality adversarial threats and environmental attenuation concerns in contested military and civilian (urban) environments (the step is directed to accessing information, which is understood to be significant extra-solution activity and is well understood, routine, and conventional activity of preparing data for presentation (MPEP 2106.05(d)(II)(i))))).
Although the additional element limits the identified judicial exceptions. The limitation merely confines the use of the abstract idea to a particular technological environment and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional element
providing a linear representation of the underlying data set which is compatible to existing deep learning (DL) model architectures for training in autonomy tasks (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g))
accessing the entire degree of freedom (DOF) space of raw multimodal sensor data for mitigating sensor modality adversarial threats and environmental attenuation concerns in contested military and civilian (urban) environments (the step is directed presenting information, which is understood to be significant extra-solution activity, and is well understood, routine, and conventional activity of preparing data for presentation (MPEP 2106.05(d)(II)(i))))).
As explained above, the additional element are recited at a high level of generality. These elements amount to collecting, evaluating, and presenting information are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II. The recitation of a computer to perform these limitations amounts to no more than mere instructions to apply the exception using a generic computer. Even when considered in combination, these additional elements represent mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
Thus, the claim is ineligible.
As per claim 2, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
constructing raw multimodal sensor data fusion manifolds (Mental Process performed in human mind using a pen and a paper (i.e. evaluation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
constructing raw multimodal sensor data fusion manifolds (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 3, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
wherein the raw multimodal sensor data fusion manifolds are a topological fingerprint and an avenue to the characteristics of the entire DOF space of the manifolds for mitigating at least one of single sensor modality adversarial threats, environmental attenuation concerns, classification tasks in autonomy, and predictive tasks in system health monitoring in contested military and commercial environments(Mental Process performed in human mind using a pen and a paper (i.e. evaluation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
wherein the raw multimodal sensor data fusion manifolds are a topological fingerprint and an avenue to the characteristics of the entire DOF space of the manifolds for mitigating at least one of single sensor modality adversarial threats, environmental attenuation concerns, classification tasks in autonomy, and predictive tasks in system health monitoring in contested military and commercial environments(the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 4, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
wherein the TDAML produced topological feature space precipitates reduced order modeling in existing DL model architectures contributing directly to its increased use in mobile computing platform applications and distributed analytical systems (Mental Process performed in human mind using a pen and a paper (i.e. evaluation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
wherein the TDAML produced topological feature space precipitates reduced order modeling in existing DL model architectures contributing directly to its increased use in mobile computing platform applications and distributed analytical systems (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 5, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
providing dynamic information in target recognition applications for collective deployments of small uncrewed aerial/maritime systems, ground vehicles, and ground personnel including by at least one of object detection through imagery and emitter data in commercial vehicles for avoidance and collision mitigation (Mental Process performed in human mind using a pen and a paper (i.e. observation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
providing dynamic information in target recognition applications for collective deployments of small uncrewed aerial/maritime systems, ground vehicles, and ground personnel including by at least one of object detection through imagery and emitter data in commercial vehicles for avoidance and collision mitigation (the step is directed to observation information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 6, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
wherein the raw heterogeneous multimodal sensor data includes one or more sensor data modality source components to store raw multimodal input data with each source component identified containing finitely many contributing devices with finitely many possible channels in each device of the same sensor modality (Mental Process performed in human mind using a pen and a paper (i.e. observation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
wherein the raw heterogeneous multimodal sensor data includes one or more sensor data modality source components to store raw multimodal input data with each source component identified containing finitely many contributing devices with finitely many possible channels in each device of the same sensor modality (the step is directed to observation information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 7, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
a memory content ingesting one or more modalities into a corresponding decision structure, wherein each modality's sample frequency is Hertz valued and sorted until a minimum sample frequency is identified from file metadata for the entire modality collection (Mental Process performed in human mind using a pen and a paper (i.e. observation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
a memory content ingesting one or more modalities into a corresponding decision structure, wherein each modality's sample frequency is Hertz valued and sorted until a minimum sample frequency is identified from file metadata for the entire modality collection (the step is directed to observation information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 8, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
storing the minimum sample frequency for compatible sample size decomposition in a Takens Embedding functional module and for the organization of a Raw Embedded Modality Data data structure (Mental Process performed in human mind using a pen and a paper (i.e. observation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
storing the minimum sample frequency for compatible sample size decomposition in a Takens Embedding functional module and for the organization of a Raw Embedded Modality Data data structure (the step is directed to observation information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 9, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
determining each ingested modality to be either time-series data or static data (Mental Process performed in human mind using a pen and a paper (i.e. evaluation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
determining each ingested modality to be either time-series data or static data (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 10, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
passing a time-series data modality to the Takens Embedding functional module which, from a given time series f(t) sample frequency producing a Takens embedding with embedding dimension D and time delay z as a sequence of vectors fi = (f(t),f(ti+),f(ti+2z), ..., f(ti + (D - 1)z)) c ]D, and producing an optimized finite dimensional Euclidean space topologically equivalent to the corresponding time-series data's dynamic system of origin and storing in the Raw Embedded Modality Structure data structure module and passing a static data directly to the Raw Embedded Modality Data data structure module (Mathematical Process performed using a pen and a paper (i.e. evaluation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
passing a time-series data modality to the Takens Embedding functional module which, from a given time series f(t) sample frequency producing a Takens embedding with embedding dimension D and time delay z as a sequence of vectors fi = (f(t),f(ti+),f(ti+2z), ..., f(ti + (D - 1)z)) c ]D, and producing an optimized finite dimensional Euclidean space topologically equivalent to the corresponding time-series data's dynamic system of origin and storing in the Raw Embedded Modality Structure data structure module and passing a static data directly to the Raw Embedded Modality Data data structure module (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 11, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
processing each Raw Embedded Modality Data module file for m E of persistent homology (PH) producing a persistence diagram (PD) in the corresponding PH dimension and each of the PD(Hg) modules, where 0 i <m, is then ingested into its corresponding Sub Diagram 1(PD(Hg)) module, where 0 i m and m is the prescribed number of dimensions of persistent homology, ingested into 9 topological metrics: 1) Persistent Entropy (PE), 2) Number of "Off Diagonal" Points (NoP), 3) the Bottleneck Distance Amplitude (Btl), 4) the q-Wasserstien Distance Amplitude (Wass), 5) The Persistence Landscape Amplitude (PL), 6) the Persistence Image Amplitude (PI), 7) the Betti Curve Amplitude (Bet), 8) the Persistence Silhouette Amplitude (Sil), and 9) the Persistence Heat Kernel (Heat) which produces a unique topological fingerprint for each ingested persistence diagram generated from the raw sensor modalities and storing as a 9 dimensional real valued vectors in the TDAML Feature Space data structure (Mathematical Process performed in human mind using a pen and a paper (i.e. evaluation)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
processing each Raw Embedded Modality Data module file for m E of persistent homology (PH) producing a persistence diagram (PD) in the corresponding PH dimension and each of the PD(Hg) modules, where 0 i <m, is then ingested into its corresponding Sub Diagram 1(PD(Hg)) module, where 0 i m and m is the prescribed number of dimensions of persistent homology, ingested into 9 topological metrics: 1) Persistent Entropy (PE), 2) Number of "Off Diagonal" Points (NoP), 3) the Bottleneck Distance Amplitude (Btl), 4) the q-Wasserstien Distance Amplitude (Wass), 5) The Persistence Landscape Amplitude (PL), 6) the Persistence Image Amplitude (PI), 7) the Betti Curve Amplitude (Bet), 8) the Persistence Silhouette Amplitude (Sil), and 9) the Persistence Heat Kernel (Heat) which produces a unique topological fingerprint for each ingested persistence diagram generated from the raw sensor modalities and storing as a 9 dimensional real valued vectors in the TDAML Feature Space data structure (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 12, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
ingesting the TDAML Feature Space data structure into the Custom DL Model functional module as a feature space for training Deep Learning (DL) models and storing a trained TDAML DL model's data structure in the decision structure functional module (Mental Process performed in human mind using a pen and a paper (i.e. judgement)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
ingesting the TDAML Feature Space data structure into the Custom DL Model functional module as a feature space for training Deep Learning (DL) models and storing a trained TDAML DL model's data structure in the decision structure functional module (the step is directed to storing information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
As per claim 13, the rejection of claim 1 is incorporated.
Step 1: The claim recites a method, which is one of the four statutory categories of eligible matter.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated, the limitation of
ingesting the TDAML Feature Space data structure into the Custom DL Model functional module as a feature space for training Deep Learning (DL) models and storing a trained TDAML DL model's data structure in the decision structure functional module (Mental Process performed in human mind using a pen and a paper (i.e. evaluating)).
Step 2A Prong 2: the judicial exceptions are not integrated into a practical application. The claim recites additional elements of
wherein the DL model is a fully connected DNN with several hidden layers containing several thousand trainable parameters trained with a random train/test split (e.g., an 80/20, 70/30, etc. type train/test split) on supervised learning for binary/multi-classification of one or more mobile targets in a designated domain (the step is directed to evaluating information, which is understood to be significant extra-solution activity, see MPEP 2106.05(g)).
The limitation recited at high level of generality and thus are insignificant extra-solution activity. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. Mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept.
The claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1 – 9 are rejected under 35 U.S.C. 103 as being unpatentable over Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration), in view of Lahat et al (Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects).
As per claim 1, Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration) discloses,
A method of topological data analysis feature engineering for data fusion and autonomy (pg.5 col.1 lines 43 – 45; “processing of multi-modal data using the multimodal DBM based feature projection, topological analysis”).
comprising: providing a Topological Data Analysis AI/ML Pipeline (TDAML) algorithm for multimodal sensor data fusion in an autonomy applications system (abstract; lines 1 – 2; “processing pipeline for fusing ‘raw’ and / or feature-level multi-sensor data” and abstract lines 18 – 19; “Deep Learning (DL), topological analysis over probability measure (TAPM)”).
further comprising: combining raw heterogeneous multimodal sensor data at the topological level (abstract; lines 41 – 44; “desire to be rapidly able to fuse between different combinations of sensor modalities, and (c) potential need to transfer information between platforms that host different sensors”).
measuring, recording, and tracking linear representations of an underlying data set (abstract; lines 45 – 48; “enables accurate, real-time target detection, tracking, and recognition of known and unknown moving or stationary targets or events and their activities evolving over space and time”).
providing a linear representation of the underlying data set which is compatible to existing deep learning (DL) model architectures for training in autonomy tasks (abstract lines 18 – 19; “Deep Learning (DL), topological analysis” and pg.4 col.1 lines 55 – 57; “TDA’s attractive properties include looking for structure in data with little or no preconception of its form”).
Garagic does not specifically disclose accessing the entire degree of freedom (DOF) space of raw multimodal sensor data for mitigating sensor modality adversarial threats and environmental attenuation concerns in contested military and civilian (urban) environments.
However, Lahat et al (Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects) in an analogous art discloses,
and accessing the entire degree of freedom (DOF) space of raw multimodal sensor data for mitigating sensor modality adversarial threats and environmental attenuation concerns in contested military and civilian (urban) environments (pg.1449 col.2 lines 13 – 17; “availability of multiple data sets that contain information, obtained using different acquisition methods, about the same system, introduces new degrees of freedom that raise questions beyond those related to analyzing each data set separately”).
Therefore, it would have been obvious to one of ordinary skill in the art before the invention was filed to incorporate degree of freedom associated multiple data sets acquire using different acquisition method of the system of Lahat into fusing of raw multi-sensor data of the system of Garagic to provide a complete knowledge of the topology data of interest.
As per claim 2, the rejection of claim 1 is incorporated and further Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration) discloses
further comprising constructing raw multimodal sensor data fusion manifolds ((abstract; lines 1 – 2; “processing pipeline for fusing ‘raw’ and / or feature-level multi-sensor data”).
As per claim 3, the rejection of claim 2 is incorporated and further Lahat et al (Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects)
wherein the raw multimodal sensor data fusion manifolds are a topological fingerprint and an avenue to the characteristics of the entire DOF space of the manifolds for mitigating at least one of single sensor modality adversarial threats, environmental attenuation concerns, classification tasks in autonomy, and predictive tasks in system health monitoring in contested military and commercial environments (pg.1454 col. lines 20 – 35; “data fusion contest of the IEEE Geoscience and Remote Sensing Society (GRSS) ….. multimodal change detection, in which the purpose is to detect changes in an area before and after an event ……multimodal multitemporal data fusion of optical, SAR, and LiDAR images taken at different years over the same urban area, where suggested applications
include assessing urban density, change detection and overcoming adverse illumination conditions for optical sensors”).
Therefore, it would have been obvious to one of ordinary skill in the art before the invention was filed to incorporate degree of freedom associated multiple data sets acquire using different acquisition method of the system of Lahat into fusing of raw multi-sensor data of the system of Garagic to provide a complete knowledge of the topology data of interest.
As per claim 4, the rejection of claim 3 is incorporated and further Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration) discloses
wherein the TDAML produced topological feature space precipitates reduced order modeling in existing DL model architectures contributing directly to its increased use in mobile computing platform applications and distributed analytical systems (pg.2 col.1 lines 16 – 19; “compact feature representations fused over multiple sensing modalities and demonstrate that these representations enable accurate realtime target detection, tracking, and recognition”).
As per claim 5, the rejection of claim 4 is incorporated and further Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration) discloses,
further comprising providing dynamic information in target recognition applications for collective deployments of small uncrewed aerial/maritime systems, ground vehicles, and ground personnel including by at least one of object detection through imagery and emitter data in commercial vehicles for avoidance and collision mitigation (pg.2 lines 22 – 26; “tracking, and identifying multiple moving vehicles with multiple distinct emitting communication devices within an area of interest can be accomplished by using simultaneous inputs from a full motion video of a scene containing these vehicles and observed by an airborne platform”).
As per claim 6, the rejection of claim 1 is incorporated and further Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration) discloses,
wherein the raw heterogeneous multimodal sensor data includes one or more sensor data modality source components to store raw multimodal input data with each source component identified containing finitely many contributing devices with finitely many possible channels in each device of the same sensor modality (abstract; lines 41 – 44; “desire to be rapidly able to fuse between different combinations of sensor modalities, and (c) potential need to transfer information between platforms that host different sensors” and pg.3 col.1 lines 11 – 12; “computed over any inferred manifold across heterogeneous data sources”).
As per claim 7, the rejection of claim 6 is incorporated and further Lahat et al (Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects) discloses,
further comprising a memory content ingesting one or more modalities into a corresponding decision structure, wherein each modality's sample frequency is Hertz valued and sorted until a minimum sample frequency is identified from file metadata for the entire modality collection (pg.1463 col.1 lines 43 – 47; “An audio signal is usually sampled at several kilohertz whereas the video signal is typically sampled at 15–60 Hz [144] (Example II-A.1). In meteorological monitoring (Example II-C.2), each modality has very distinct spatial and temporal resolutions”).
As per claim 8, the rejection of claim 7 is incorporated and further Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration) discloses,
further comprising storing the minimum sample frequency for compatible sample size decomposition in a Takens Embedding functional module and for the organization of a Raw Embedded Modality Data data structure (pg.6 col.1 lines 11 – 14; “multi-source separation algorithm enables us to estimate transmitted frequency corresponding to each tracked vehicle, resulting in a more accurate estimate of the Doppler shift corresponding to an emitter”).
As per claim 9, the rejection of claim 8 is incorporated and further Garagic et al (Upstream Fusion of Multiple Sensing Modalities Using Machine Learning and Topological Analysis: An Initial Exploration) discloses,
further comprising determining each ingested modality to be either time-series data or static data (pg.4 col.2 lines 45 – 46; “a time series of data, perhaps multi-modal, divides time into small windows”).
Conclusion
7. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
TITLE: Simultaneous Localization and Mapping (SLAM) and Data Fusion in Unmanned Aerial Vehicles: Recent Advances and Challenges, March 2022, Gupta et al.
TITLE: A topological analysis of cointegrated data: a Z24 Bridge case study, September 2022, author: Gowdridge et al.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AUGUSTINE KUNLE OBISESAN whose telephone number is (571)272-2020. The examiner can normally be reached 9:00am - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ajay Bhatia can be reached at (571) 272-3906. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AUGUSTINE K. OBISESAN/
Primary Examiner
Art Unit 2156
7/19/2026
Prosecution Insights