DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-12 have been presented for examination based on the amendment filed on
12/16/2022.
Claims 1-12 are objected under 35 U.S.C. 112(f)
Claim 1-12 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
Claims 1-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Claim(s) 1-12 rejected under 35 U.S.C. 103 as being unpatentable over US PGPUB No. US11436484B2 by Farabet et al. view of US PGPUB No. US20220366568A1 by Arar et al.
This action is made Non-Final.
---- This page is left blank after this line ----
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Three-Prong Test Analysis for "Circuit" Limitations
1. Prong 1: Use of "Means," "Step," or a Generic Placeholder
The claim limitations use the term "circuit" (specifically, "data management circuit" and "participation system management circuit"). While "circuit" is traditionally a structural term, the USPTO and the courts treat it as a generic placeholder (nonce term) when it does not denote a specific structural device to a person of ordinary skill in the art (POSITA) and is instead used to recite a function. In the context of this disclosure, "circuit" serves as a substitute for "means" because it identifies a functional unit rather than a physical arrangement of specific electronic components.
2. Prong 2: Modified by Functional Language
The generic placeholders are modified by functional language linked by the phrase "configured to":
Data management circuit: "configured to generate data sets for each execution mode... and for each test environment...".
Participation system management circuit: "configured to distribute... the data sets corresponding to the execution mode and the test environment" and "configured to control the test participation system".
This functional language describes what the elements do (generate, distribute, control) rather than what they are.
3. Prong 3: Absence of Sufficient Structure
The terms are not modified by sufficient structure, material, or acts for performing the claimed functions.
The claim does not recite specific structural modifiers (e.g., "logic circuit" or "voltage comparator circuit") that would provide a name for a known structure.
For computer-implemented functions such as "generating data sets" and "distributing data," the required structure must be an algorithm (e.g., a flowchart or prose sequence of steps) that transforms a general-purpose processor into a special-purpose circuit.
Because the claim recites the functional result without any structural or algorithmic limitations, the presumption that 112(f) does not apply is overcome.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: "circuit" (specifically, "data management circuit" and "participation system management circuit") in claims 1,5-6.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-12 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. While the specification references these elements generally in Figures 1, 2, and 4, and in the corresponding detailed description, the disclosure is limited to a mere restatement of the functional results the circuits are intended to achieve .For computer-implemented functional claim limitations, the specification must disclose the algorithm (e.g., the finite sequence of steps, procedures, or flowcharts) that performs the claimed function in sufficient detail so that one of ordinary skill in the art would understand how the inventor intended the function to be performed. The specification lacks any disclosure of the specific steps or logic flow used by the "data management circuit" to generate mode-specific data sets or by the "virtual model generation circuit" to produce a virtual model. Merely identifying a "circuit" and its intended "result" is insufficient to show possession of the invention. The description of a single result (e.g., "controlling the test participation system") does not entitle the inventor to claim any and all means or circuits for achieving that objective. Without a disclosed correlation between the function and a specific structure or acts, the disclosure is "little more than a wish for possession" and fails to satisfy 35 U.S.C. 112(a). Regarding Claims 3 and 4, which recite a broad genus of "test environments" (development, verification, and operation systems), the specification fails to describe a representative number of species or the structural/logical traits common to these varied systems to support the full scope of the genus.
Claims 1-12 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention.
Determination: Not Enabled. A person of ordinary skill in the art (POSITA) would require undue experimentation to construct the claimed "circuits" from the functional flow-style steps provided.
1. Quantity of experimentation: High. A POSITA must design the entire hardware/logic architecture for the "circuits" as none is provided.
2. Amount of guidance provided: Low. The spec provides high-level functional results (S420–S450) but lacks "how-to" guidance for implementation.
3. Working examples: None. No specific code, circuit diagrams, or physical prototypes are described.
4. Nature of the invention: This is a computer-implemented/automotive testing system, which typically requires precise logic and interface protocols.
5. State of the prior art: While testing is known, the specific automated "management circuit" interactions claimed are not shown to be well-known enough to omit structural detail.
6. Level of ordinary skill: A skilled engineer in automotive systems or computer architecture.
7. Predictability of the art: Low to Medium. Converting broad functional goals into operative "circuits" without an algorithm is unpredictable.
Breadth of the claims: Very Broad. "Each execution mode" and "each test environment" covers infinite variations not taught in the spec.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim limitation “Circuit” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification fails to disclose the corresponding structure (algorithm) for the computer-implemented functions performed by the "data management circuit" and "participation system management circuit." While the specification describes "steps" such as S410 (generating) and S420 (distributing), these steps merely restate the functional results and do not provide the underlying logic or sequence of steps required to perform the functions. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over US PGPUB No. US11436484B2 by Farabet et al. as applied to claims above, and further in view of US PGPUB No. US20220366568A1 by Arar et al.
Regarding Claim 1
Farabet et al. teaches (L1) A test management system comprising: (Farabet et al. [§9 Line:20, Fig.4A-C] : “The simulation system 400---e.g., represented by simulation systems 400A, 400B, 400C, and 400D, described in more detail herein-may generate a global simulation that simulates a virtual world or environment (e.g., a simulated environment) that may include artificial intelligence (AI) vehicles or other objects (e.g., pedestrians, animals, etc.), 25 hardware-in-the-loop (HIL) vehicles or other objects, soft- ware-in-the-loop (SIL) vehicles or other objects, and/or person-in-the-loop (PIL) vehicles or other objects..” The examiner interprets where the test management system is shown as the simulation system. and validation of autonomous driving software stacks.)
PNG
media_image1.png
702
1080
media_image1.png
Greyscale
PNG
media_image2.png
718
1066
media_image2.png
Greyscale
PNG
media_image3.png
810
1130
media_image3.png
Greyscale
(L2) a data management circuit1 {Fig. 1 [100]) configured to generate data sets (Farabet et al [§5¶3]: “As such, the system 100 may include a re-simulation system that uses physical sensor data generated by vehicle(s) 102 in real-world environments to train, test, verify, and/or validate one or more DNNs for use in the software stack(s) 116..” The examiner interprets where the data management circuit is shown as the system and the generated data set is shown as physical sensor data) for each execution mode2 [Farabet et al [Fig.1 & Fig. 4A-C] of a test management system 3
PNG
media_image4.png
812
1168
media_image4.png
Greyscale
(L3) and a participation system management circuit configured to distribute, to the test participation system,4 the data sets corresponding to the execution mode and the test environment. 5 (Farabet et al. [§18 Line:16-29]: “In some examples, the vehicle simulator component(s) 406 may include a network interface (e.g., one or more network interface cards (NICs) 450) that may simulate or emulate RADAR sensors, LIDAR sensors, and/or IMU sensors (e.g., by providing 8 Gigabit ports with precision time protocol (PTP) support). In addition, the vehicle simulator component(s) 406 may include an input/output (I/O) analog integrated circuit. Registered Jack (RJ) interfaces (e.g., RJ45), high speed data (HSD) inter- faces, USB interfaces, pulse per second (PPS) clocks, Ethernet (e.g., 10 Gb Ethernet (GbE)) interfaces, CAN interfaces, HDMI interfaces, and/or other interface types may be used to effectively transmit and communication data between and among the various component(s) of the system...” The examiner interprets where the participation system management circuit is shown as the vehicle simulator, the network interrace and (I/O) analog integrated circuit as shown as a test participation system, and the distribution of data sets is shown as transmitting and communicating data.)
Farabet failed to disclose a data management circuit6 configured to generate data sets of a test management system explicitly for each test environment7 of a test participation system and deployment logic where models and data are selected for specific deployment environments. However, Arar et al teaches a data management circuit8 configured to generate data sets of a test management system explicitly for each test environment9 of a test participation system. (Arar et al. [Abstract]: “In various examples, an adaptive eye tracking machine learning model engine ("adaptive-model engine") for an eye tracking system is described. The adaptive-model engine pipeline (adaptive-model training pipeline”) that supports collecting data, training, optimizing, and deploying an adaptive eye tracking model that is a customized eye tracking model based on a set of features of an identified deployment environment…” The examiner interprets where the test environment is shown as a deployment environment. Arar et al. Fig1A:
PNG
media_image5.png
850
1260
media_image5.png
Greyscale
) Arar et al. further describes a participation system management circuit configured to distribute, to the test participation system,10 the data sets corresponding to the execution mode and the test environment. (Arar et al. [Fig.1B] :
PNG
media_image6.png
858
996
media_image6.png
Greyscale
)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The combination of Farabet et al. and Arar et al. is merely the predictable use of prior art elements according to their established functions to yield an improved, systematic testing workflow. A POSITA would have had a reasonable expectation of success in combining these features, as both references address the same fundamental problem of ADS validation across varied technological domains.
Regarding Claim 2
Farabet et al. in combination with Arar et al teaches (L1) The test management system of claim 1 (See Claim 1). Farabet et al. teaches (L2) wherein the test participation system comprises an actual vehicle for a test (See “Vehicle hardware” Farabet et al. [See Fig.4B in claim 1]: The examiner interprets where the actual vehicle is shown as vehicle hardware.)
(L3) and a virtual model generated for a test.
(Farabet et al. [Abstract]: “In various examples, physical sensor data may be generated by a vehicle in a real-world environment. The physical sensor data may be used to train deep neural networks (DNNs). The DNNs may then be tested in a simulated environment-in some examples using hardware configured for installation in a vehicle to execute an autonomous driving software stack-to control a virtual vehicle in the simulated environment or to otherwise test, verify, or validate the outputs of the DNNs. Prior to use by the DNNs, virtual sensor data generated by virtual sensors within the simulated environment may be encoded to a format consistent with the format of the physical sensor data generated by the vehicle.”
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The motivation for this combination is to allow for safe, scalable validation where software can be tested faster than real-time in a virtual model before being verified in a high-fidelity environment using an actual vehicle. The use of a hybrid participation system (actual + virtual) was a well-known expedient in the art to ensure "bit-to-bit" consistency between simulation and reality. The result is the predictable integration of known testing targets into a unified management framework.
Regarding Claim 3
Farabet et al. in combination with Arar et al. teaches the (L1) test management system of claim 1 (See claim 1). Farabet et al. teaches (L2) (L3) and a simulation mode. (Farabet et al. [See Fig.1 & Fig.4A-C in claim 1] [§11 Line: 4 “The simulation system 400A may generate a simulated environment 410 that may include AI objects 412 (e.g., AI objects 412A and 412B), HIL objects 414, SIL objects 416, PIL objects 418, and/or other object types.”)
Farabet et al. fails to disclose wherein the execution mode explicitly comprises an actual environment test mode. However, Arar et al. teaches wherein the execution mode comprises an actual environment test mode (Arar et al. [See Fig.1A in claim 1] The examiner interprets an actual environment test mode as shown in "real-time testing (in-car)" and “in-car setup".
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The motivation for this combination is to allow for safe and scalable validation where software is initially verified in a simulation mode to identify failures before being tested in an actual environment test mode. The use of these dual modes was a well-known expedient in the art to ensure "bit-to-bit" consistency and safety for automated driving systems. The result is the predictable integration of known testing environments into a unified execution framework.
Regarding Claim 4
Farabet et al in combination with Arar et al. teaches (L1) The test management system of claim 1 (See claim 1). Farabet et al. teaches (L2) wherein the test environment comprises a development system11, (See Fig.1 of Farabet et al. in claim 1: The examiner interprets the development system as shown a training sub-system 106) (See Fig.A1. of Arar et al. in claim 1: Examiner interprets the development system as shown in the pipeline comprising data collection 110, data preparation 120, and model training 140. (L3) a verification system12, (See Fig.1 of Farabet et al. in claim 1) (L4) and an operation system13. (See Fig,1 of Farabet et al. claim 1: The examiner interprets the operation system as shown in the deployment of validated software stacks to vehicle hardware for execution in autonomous vehicle.). While Farabet et al. teaches all limitations, Farabet et al. fails to explicitly organizes these environments. However, Arar et al teaches wherein the test environment comprises 1415, (See Fig.1A of Arar et al. in claim 1: The examiner interprets the verification system as shown as a multi-stage validation.) and an operation system16. (See Fig.1A of Arar et al. in claim 1: The examiner interprets the operation system as shown in a driving system.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The motivation for this combination is to provide a structured, end-to-end validation workflow where software can be predictably improved through training, verified for safety in a lab, and monitored during real-world operation. The use of these three functional systems was a well-known expedient in the art of safety-critical software engineering. The result is merely the predictable integration of known development stages into a unified test management framework.
Regarding Claim 5
Farabet et al. in combination with Arar et al. teaches (L1) The test management system of claim 1 (See claim 1).
Farabet et al. teaches (L2) further comprising a time setting circuit17 (Farabet et al. [Fig.4E]:
PNG
media_image7.png
842
1216
media_image7.png
Greyscale
The examiner interprets where the time setting circuits is shown as SYNC Components.) configured to periodically set, correct, and distribute time information (Farabet et al [§18 Line:22]: “In addition, the vehicle simulator component(s) 406 may include an input/output (I/O) analog integrated circuit. Registered Jack (RJ) interfaces (e.g., RJ45), high speed data (HSD) interfaces, USB interfaces, pulse per second (PPS) clocks, Ethernet (e.g., 10 Gb Ethernet (GbE)) interfaces, CAN interfaces, HDMI interfaces, and/or other interface types may be used to effectively transmit and communication data between and among the various component(s) of the system”.. The examiner interprets where periodic setting, correcting, and distributing of time information to the data sets is shown pulse per second (PPS) clocks, Ethernet (e.g., 10 Gb Ethernet (GbE)) interfaces, and CAN interfaces transmitting and communicating data among the various component(s) of the system.) to the data sets. (Farabet et al. [Fig.7]:
PNG
media_image8.png
692
1002
media_image8.png
Greyscale
The examiner interprets the datasets as shown as the sensor data stream in the data flow diagram encoded for a current time slice.
Farabet et al. [Fig.6A]:
PNG
media_image9.png
820
1184
media_image9.png
Greyscale
The examiner interprets the datasets is shown as software stack encoded with timestamps. Farabet et al. fails to explicitly disclose the "periodic" nature of setting and correcting time information within the data set. However, Arar et al. teaches further comprising a time setting circuit18 configured to periodically set, (Arar et al. [0049]: “In this regard, processing surround scene data of a set of surround scene data types from the bench setup and the in-vehicle setup is based at least on two or more common data processors, while at least one different data processor of the bench setup and the 1n-vehicle setup processes surround scene data and executes data synchronization operations exclusively for the bench setup or the in-vehicle setup.”
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The motivation for this combination is to ensure that when data is distributed to different "test participation systems" (as in Farabet et al.), they remain aligned in time, which is essential for the bit-to-bit correct signal integrity that Farabet et al. claims is vital for safety validation. The use of such circuits (e.g., NTP servers or precision time protocols) to maintain data consistency in distributed networks was a well-known expedient in the art of software engineering and ADS validation. The result is merely the predictable integration of known synchronization methods into a comprehensive test management framework.
Regarding Claim 6
Farabet et al. in combination with Arar et al. teaches (L1) The test management system of claim 1 (See clam 1). Farabet et al. teaches (L2) wherein: the test participation system comprises a virtual model, (See claim 2 [L3]) and the test management system further comprises a virtual model generation circuit19 (Farabet et al. [§24 Line:27 “The simulation engine 630 may then update the object data and characteristics (e.g., within the asset data store(s) 636), may update the GI (and/or other aspects such as reflections, shadows, etc.), and then may generate and provide updated sensor inputs to the GPU platform 624. This process may repeat until a simulation is completed.” The examiner interprets where the virtual model generation circuit is shown as the simulation engine. configured to generate the virtual model. (See Farabet et al. §9 Line:20 in claim 1: The examiner interprets the generation of the virtual model is shown as the generation of a global simulation that simulates a virtual world or environment) (Farabet et al [Fig.10, B1012]:
PNG
media_image10.png
1440
788
media_image10.png
Greyscale
Farabet et al fails to explicitly disclose a "virtual model generation circuit" for creating these models. However, Arar et al teaches the test management system further comprises a virtual model generation circuit20 configured to generate the virtual model.
(See Arar et al. Fig.1A in claim 1: The examiner interprets the generation of the virtual models is shown as the generation of synthetic data.) (See Arar et al. Fig.1B in claim 1: The examiner interprets the generation of a virtual model is shown as the automatic generation of adaptive tracking models [virtual models] customized to a specific environment)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The motivation for this combination is to automate the instantiation of test actors, thereby allowing the system to scale its verification with computing resources and perform rapid generation of test permutations without the need for manual model creation. The result is merely the predictable use of an automated generation engine within a known simulation framework to solve the known problem of test-case scalability in ADS validation.
Regarding Claim 7
Farabet et al teaches (L1) A method of a test, comprising: (Farabet et al. §3 Line:34 “Systems and methods disclosed are related to training, testing, and verifying autonomous machines or objects in simulated environments.”) (L2) generating data sets for each execution mode of a test management system a mode of which is a setting mode21and for each test environment of a test participation system; (Farabet et al. [Fig.3A-C]:
PNG
media_image11.png
700
1076
media_image11.png
Greyscale
PNG
media_image12.png
690
1082
media_image12.png
Greyscale
PNG
media_image13.png
686
1018
media_image13.png
Greyscale
The examiner interprets where the setting mode is shown where the data is ingested, curated, and labeled in the workflows [300A-C] to generate, train, and test data sets for specific modes such as re-simulation or simulation [Fig.1 & Fig.4A-F {D & F Below}]:
PNG
media_image14.png
660
546
media_image14.png
Greyscale
(L3) distributing, to the test participation system, the data sets corresponding to the execution mode and the test environment; (See Farabet et al. [Fig.7]: The examiner interprets where the test participation system is shown as the vehicle hardware and the distribution of corresponding data sets is shown as the workflow of sensor data streams and object states based on the current simulation slices.) (L4) changing the mode of the test management system into an execution mode; (See Farabet et al. [Fig.1 & Fig.4A-F in previous claims]: the examiner interprets where changing the mode is shown as switching between a training/curation state and active re-simulation or simulation state.) (L5) performing, by the test participation system, a test; and (Farabet et al. [§24 Line:8 “In examples where the virtual objects are simulated using HIL objects, the sensor inputs may be provided to the vehicle hardware 104 which may use the software stack(s) 116 to perform one or more operations and/or generate one or more commands, such as those described herein… The one or more operations or commands may be transmitted to the simulation engine 630 which may update the behavior of one or more of the virtual objects based on the operations and/or commands.” The examiner interprets where the test is shown as the software stack being executed on the vehicle hardware or HIL object to generate driving commands and respond to the simulated environment. (L6) storing results of the test derived by the test participation system. (Farabet et al. §25 Line:3 “Logs 806 may be generated and passed to re-simulator/simulator 804. The re-simulator/simulator 804 may provide sensor data to the software stack(s) 116 which may be executed using HIL, SIL, or a combination thereof. The KPI evaluation component 802 may use different metrics for each simulation or re-simulation instance. For examples, for re-simulation, KPI evaluation component may provide access to the original re-played CAN data and/or the 10 newly generated CAN data from the software stack(s) 116 (e.g., from HIL or SIL). In some examples, performance could be as simple as testing that the new CAN data does not create a false positive-such as by triggering Automatic Emergency Braking (AEB), or another ADAS functionality.” The examiner interprets where storing results is shown as storing logs of simulation data and using KPI evaluation components to produce reports on software performance. While Farabet et al teaches managing real-world and simulated sensor data for validation and discloses a simulation host and management device configured to distribute data to targeted hardware components; Farabet et al fails to explicitly teach the systematic generation, and segregation of data sets tailored for specific validation setups (test environments) and further describe selection and deployment logic where models and data are deployed to specific environments. However, Arar et al teaches (L2) generating data sets for each execution mode of a test management system a mode of which is a setting mode22and for each test environment of a test participation system; See Arar et al. [Fig.1A] : The examiner interprets where the test environment is shown as bench setup vs. in-car setup. (L3) distributing, to the test participation system, the data sets corresponding to the execution mode and the test environment; (See Fig.1B of Arar et al in Claim 1: The examiner interprets where the test participation system is shown as specific deployment environments and the distribution of data sets is shown as the transmission of adapted models.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The combination of Farabet et al. and Arar et al. is merely the predictable use of prior art elements according to their established functions to yield an improved, systematic testing workflow. A POSITA would have had a reasonable expectation of success in combining these features, as both references address the same fundamental problem of automated driving system (ADS) validation across varied technological domains.
Regarding Claim 8
Farabet et al in combination with Arar et al. teaches The method of claim 7, (See claim 7) wherein: the execution mode comprises an actual environment test and a simulation mode, (Method version of claim 3. Rejection in the same manner.) and the test environment comprises a development system, a verification system, and an operation system. (Method version of claim 4. Rejection in the same manner.)
Regarding Claim 9
Farabet et al in combination with Arar et al. teaches The method of claim 7, (See claim 7)
wherein the test participation system comprises an actual vehicle for a test and a virtual model generated for a test. (Method version of claim 2. Rejection in the same manner.)
Regarding Claim 10
Farabet et al. in combination of Arar et al. teaches The method of claim 7, (see Claim 7)
wherein: the test participation system comprises a virtual model, and the method further comprises generating the virtual model. (Method version of claim 6. Rejection in the same manner.)
Regarding Claim 11
Farabet et al. in combination with Arar et al. teaches (L1) The method of claim 7, (See Claim 7) (L2) wherein the mode of the test management system is configured to be set as a setting mode, (Farabet et al. [§6 Line: 1]: “The process 118 may further include data indexing and curation 124, the data labeling services 126, modeling training 128, model refinement, pruning, and/or fine tuning 130, model validation 132, and/or updating global coverage maps 134.” The examiner interprets where the setting mode is shown as where the data indexing, curation, and labeling occur prior to active simulation.
(L3) an execution mode, (See Farabet et al. Fig.6A in claim 5: The examiner interprets an execution mode shown as the illustration of a simulation system at runtime.) (L4) or a reproduction mode. (See Farabet et al. Fig.1 in claim 1: The examiner interprets a reproduction mode as shown in re-simulation mode. See Farabet et al. §6 Line: 35 “Once the datasets are stored, the datasets may be used and re-used to reproduce training results exactly, or run and re-run simulation jobs.” And Farabet et al. §7 Line: 7 “As such, the platform may enable systematic tracking of all information pertaining to experiments, thereby enabling reproducible experiments. While Farabet et al. discloses operational workflows for "simulation" (corresponding to an execution mode) and re-simulation (corresponding to a reproduction mode), Farabet et al. fails to explicitly teaches a Multi-Stage Validation pipeline featuring a Data Preparation phase (120) specifically intended to generate and segregate data for specific validation setups. However, Arar et al. teaches (L2) wherein the mode of the test management system is configured to be set as a setting mode, (See Arar et al Fig.1A in claim 1: The examiner interprets setting mode as shown in the data preparation stage 120 involving data extracting, filtering, and labeling,)
(L3) an execution mode(See Arar et al Fig,1A in claim 1: The examiner interprets an execution mode as shown in real-time testing [in-car] stage in its multistage validation pipeline.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The motivation for this combination is to provide a structured, end-to-end validation workflow where data can be predictably configured, executed to identify failures, and then reproduced for benchmarking and error analysis. The categorization of system states into these three modes was a well-known expedient in the art of safety-critical software engineering to ensure "bit-to-bit" consistency and error traceability. The result is merely the predictable integration of known operational stages into a unified test management method.
Regarding Claim 12
Farabet et al. teaches (L1) The method of claim 11, (See Claim 11) (L2) wherein the test management system is configured to be changed from the setting mode to the execution mode or the reproduction mode and (See Farabet et al. Fig.7 in claim 5: The examiner interprets the setting mode as shown in the workflow 118 where the data is ingested and curated, the execution mode shown as the active simulation, and the reproduction mode as shown as re- simulation.) (L3) changed from the execution mode or the reproduction mode to the setting mode. (Farabet et al. §6 Line:57 “ Model refinement, pruning, and/or fine tuning 130 may include updating the DNNs to further refine and improve the accuracy and efficacy of the DNNs. For example, hyper- parameter discovery may be enabled by an experiment service that may track information on the hyper-parameter space to explore hyper-parameter configurations, metrics, and model versions generated by each experiment” The examiner interprets changed from execution mode or reproduction mode to setting mode as shown in the aftermath of validation [execution, see prior (L2)] and re-simulation [reproduction, see prior (L2)], the system performs model refinement, pruning, and/or finetuning which requires returning to the data curation and training phases [setting mode, see prior (L2)]. While Farabet et al. teaches a data flow starting from a training sub-system (setting mode) and proceeding to a validation/re-simulation sub-system (execution/reproduction mode); Farabet et al. fails to teach a pipeline where Data Preparation 120 occurs before Multi-Stage Validation 150 and explicitly teaches that the validation stage generates Feedback (Benchmarking & Error Analysis) 160 which is provided back to the Data Preparation 120 and Data Collection 110 stages. However Arar et al teaches (L2) wherein the test management system is configured to be changed from the setting mode to the execution mode or the reproduction mode and(See Arar et al Fig,1A in claim 1: The examiner interprets an execution mode as shown in real-time testing [in-car] stage in its multistage validation pipeline) (L3) changed from the execution mode or the reproduction mode to the setting mode. (See Arar et al. Fig.1A in claim 1: The examiner interprets changed from the execution mode to setting mode as shown in in the feedback loop 160 from validation (execution) results back to data preparation (setting mode).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teaching of Arar with the system of Farabet. The motivation for this combination is to provide a structured, end-to-end validation loop where errors identified in an "execution" or "reproduction" phase can be immediately addressed by returning the system to a "setting" mode for data refinement. The use of such feedback loops is a routine and well-known practice in the art of machine learning and safety-critical software validation to ensure bit-to-bit consistency and error traceability,. The result is merely the predictable integration of a known feedback mechanism into a comprehensive test management framework.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARIC RAYJEE MARKS whose telephone number is (571)467-6372. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARIC R MARKS/Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
1 Spec [0045] “Accordingly, the data management circuit 110 that generates a data set may generate data sets for the actual environment test mode and the simulation mode, respectively, and the data management circuit 10 may separately manage the data sets. The data management circuit 110 may generate data sets for test environments within the actual environment test mode and the simulation mode, respectively, and may separately manage the data sets.”
2 Spec [0010] “In the test management system, the execution mode may include an actual environment test mode and a simulation mode.”
3 Spec [0011] “In the test management system, the test environment may include a development system, a verification system, and an operation system.”
4 Spec [0021] “In the method of managing a test management system, the test participation system may include an actual vehicle for a test and a virtual model generated for a test.”
5 Spec [0008] “the system also includes a participation system management circuit configured to distribute, to the test participation system, the data sets corresponding to the execution mode and the test environment and configured to control the test participation system.”
6 Spec [0045] “Accordingly, the data management circuit 110 that generates a data set may generate data sets for the actual environment test mode and the simulation mode, respectively, and the data management circuit 10 may separately manage the data sets. The data management circuit 110 may generate data sets for test environments within the actual environment test mode and the simulation mode, respectively, and may separately manage the data sets.”
7 Spec [0011] “In the test management system, the test environment may include a development system, a verification system, and an operation system.”
8 Spec [0045] “Accordingly, the data management circuit 110 that generates a data set may generate data sets for the actual environment test mode and the simulation mode, respectively, and the data management circuit 10 may separately manage the data sets. The data management circuit 110 may generate data sets for test environments within the actual environment test mode and the simulation mode, respectively, and may separately manage the data sets.”
9 Spec [0011] “In the test management system, the test environment may include a development system, a verification system, and an operation system.”
10 Spec [0021] “In the method of managing a test management system, the test participation system may include an actual vehicle for a test and a virtual model generated for a test.”
11 Spec [0058]: “The development system 21 may mean an environment corresponding to a step of developing a test system for a vehicle.”
12 Spec [0058] “The verification system 222 may mean an environment corresponding to a step of verifying whether a test system operates normally by simulating the test system developed in the development system.”
13 Spec [0004] “The operation system may mean an environment corresponding to a step of actually operating a system verified in the verification system.”
14 Spec [0058]: “The development system 21 may mean an environment corresponding to a step of developing a test system for a vehicle.”
15 Spec [0058] “The verification system 222 may mean an environment corresponding to a step of verifying whether a test system operates normally by simulating the test system developed in the development system.”
16 Spec [0004] “The operation system may mean an environment corresponding to a step of actually operating a system verified in the verification system.”
17 Spec [0046] “..the time setting circuit may distribute time information for the time when the test was performed to a data set corresponding to the results of the test.”
18 Spec [0046] “..the time setting circuit may distribute time information for the time when the test was performed to a data set corresponding to the results of the test.”
19 Spec [0013] “The test management system may further Include a Virtual model generation circuit configured to generate the virtual model.”
20 Spec [0013] “The test management system may further Include a Virtual model generation circuit configured to generate the virtual model.”
21 Spec [0052} “The setting mode 211 may be a mode in which the test management system distributes, to the test participation system, data sets corresponding to the execution mode 212 and the test environment 220.”
22 Spec [0052} “The setting mode 211 may be a mode in which the test management system distributes, to the test participation system, data sets corresponding to the execution mode 212 and the test environment 220.”
Prosecution Insights