DETAILED ACTION
Status of the Claims
This office action is submitted in response to the amendment filed on 4/7/26.
Examiner notes that this application claims priority from provisional application 63663446.
Examiner further notes Applicant’s priority date of 6/24/24, which stems from the aforementioned provisional application.
Claims 1, 3-5, 7-8, 11, and 13-20 have been amended.
Therefore, claims 1-20 are currently pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1–20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Step 1: Claims 1–20 are directed to patent-eligible subject matter categories under 35 U.S.C. § 101. See MPEP § 2106.03. Specifically, claim 1 recites a portable monitoring system, and thus falls within the "machine" category. Claim 13 recites a method, and thus falls within the "process" category. Claim 16 recites a wearable or portable monitoring system, and thus falls within the "machine" category. Claims 2–12 depend from claim 1, and thus likewise fall within the "machine" category. Claims 14–15 depend from claim 13, and thus likewise fall within the "process" category. Claims 17–20 depend from claim 16, and thus likewise fall within the "machine" category. Accordingly, the claims satisfy Step 1.
Step 2A, Prong One: Independent claims 1, 13, and 16, in part, describe an invention comprising: accessing a technical specification associated with a maintenance event of an aircraft, the technical specification identifying completion thresholds indicating how each step is to be completed by an operator; obtaining data in two or more different types of modalities associated with actions taken by the operator and characteristics of the aircraft during one or more steps of the maintenance event; analyzing the data to correlate data associated with the actions taken by the operator with data associated with the characteristics of the aircraft; and examining the actions taken by the operator in real-time during each step relative to the completion thresholds to determine that each step meets the corresponding completion threshold. As such, the invention is directed to the abstract idea of monitoring and evaluating an operator's performance of maintenance procedures against defined requirements, which, pursuant to MPEP § 2106.04(a)(2), is aptly categorized as both a mental process (activities that can be performed mentally or with pen and paper by a human supervisor consulting a maintenance manual, observing the operator, and evaluating whether each step was performed correctly) and a method of organizing human activity (managing maintenance quality control by evaluating whether an operator's actions satisfy procedural requirements). Therefore, under Step 2A, Prong One, the claims recite a judicial exception.
Next, the aforementioned claims recite additional elements that are associated with the judicial exception, including: receiving data from plural sources during the maintenance event; and communicating an alert via a display device responsive to determining that an action does not meet a correct action threshold, the alert including a corrective action recommendation. Dependent claims 4–5, 7–8, 15, and 18–19 further describe communicating notifications, alerts, instructions, and approval to the operator via an output or display device. The Examiner understands these limitations to be insignificant extra-solution activity — i.e., pre-solution data gathering (receiving sensor data) and post-solution output (communicating alerts, notifications, instructions, corrective action recommendations, and approval to the operator). See Accenture Global Servs., GmbH v. Guidewire Software, Inc., 728 F.3d 1336, 108 U.S.P.Q.2d 1173 (Fed. Cir. 2013), citing Cf. Diamond v. Diehr, 450 U.S. 175, 191–192 (1981) ("Insignificant post-solution activity will not transform an unpatentable principle into a patentable process.").
The aforementioned claims also recite additional elements including: a "control unit" including one or more "processors"; a "communication system" configured to establish wireless connections with plural sources; a "body" configured to be worn by or coupled to an operator (claim 16); and a "display device" (claim 16). Dependent claims 4, 8, and 11 further describe an "output device" coupled with the control unit and sensors coupled with a wearable body. These limitations are recited at a high level of generality and appear to be nothing more than generic computer and hardware components used to apply the abstract idea. Claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 223 (2014), 110 USPQ2d 1977, 1983 (2014).
Dependent claim 12 further recites that the control unit is "an artificial intelligence (AI) or machine-learning system." The use of an AI or machine-learning system to receive data, process that data, and output a compliance determination amounts to mere instructions to implement the abstract idea on a computer — i.e., it is an "apply it" limitation. The AI or machine-learning system is simply used as a tool to carry out the abstract idea and does not add anything beyond the abstract idea itself. See MPEP § 2106.05(f); Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983.
Step 2A, Prong Two: Looking at the elements individually and in combination, the claims as a whole do not integrate the judicial exception into a practical application because they fail to: improve the functioning of a computer or a technical field; apply the judicial exception in the treatment or prophylaxis of a disease; apply the judicial exception with a particular machine; effect a transformation or reduction of a particular article to a different state or thing; or apply the judicial exception beyond generally linking the use of the judicial exception to a particular technological environment. Rather, the claims merely use a computer as a tool to perform the abstract idea, and/or add insignificant extra-solution activity to the judicial exception, and/or generally link the use of the judicial exception to a particular technological environment (e.g., aircraft maintenance monitoring using wireless sensors and wearable devices). Accordingly, the claims do not integrate the judicial exception into a practical application, and the analysis proceeds to Step 2B.
Step 2B: The claims do not include additional elements sufficient to amount to significantly more than the judicial exception. The additional elements, when considered individually and as an ordered combination, do not amount to significantly more than the abstract idea itself. Furthermore, looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or any other technology, and their collective functions are merely facilitated by generic computer implementation.
Additionally, pursuant to the requirement under Berkheimer v. HP Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018), the following citations are provided to demonstrate that the additional elements, identified above as extra-solution activity, are well-understood, routine, and conventional. See MPEP § 2106.05(d).
Receiving or transmitting data over a network. Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network).
Thus, taken alone and in combination, the additional elements do not amount to significantly more than the above-identified judicial exception, and claims 1–20 are ineligible under 35 U.S.C. § 101.
Next, claims 2–12 depend from claim 1, claims 14–15 depend from claim 13, and claims 17–20 depend from claim 16, and include all limitations contained therein. These claims do not recite any additional elements sufficient to transform the abstract idea into patent-eligible subject matter. Specifically:
Claims 2 and 17 further specify the types of modalities (video, text, audio, image, or touch) or that the control unit examines the actions based on each modality of data. These limitations add further detail regarding the categories of data collected and analyzed, amounting to additional non-structural details that further define the contours of the data analysis without meaningfully limiting the abstract idea.
Claims 3 and 14 further specify that the control unit examines the data to determine if the operator's actions are correct or incorrect based on the technical specification. These limitations further describe the evaluation and determination steps of the abstract idea without reciting any improvement to computer functionality or any other technical advance beyond the abstract idea itself.
Claims 4 and 8 further recite an output device operably coupled with the control unit, and that the control unit communicates an alert or notification to the operator when actions are determined to be incorrect or do not meet a completion threshold. The output device is a generic hardware component, and communicating a notification to the operator is well-known extra-solution activity as addressed in the Berkheimer analysis above.
Claim 5 further specifies that the control unit communicates instructions to assist the operator in completing maintenance steps. Communicating instructions to an operator is well-known extra-solution activity as addressed in the Berkheimer analysis above.
Claim 6 further specifies that the control unit confirms completion of maintenance steps. Confirming completion is part of the abstract idea of evaluating step performance against procedural requirements.
Claims 7 and 15 further specify that the control unit identifies that at least one step did not meet the completion threshold or that the operator's actions are incorrect, and communicates a notification. Identifying non-compliance is part of the abstract idea, and communicating a notification is well-known extra-solution activity as addressed in the Berkheimer analysis above.
Claim 9 further specifies that the notification includes a corrective action recommendation for the operator to remedy the step. Specifying the content of a notification to the operator is well-known extra-solution activity as addressed in the Berkheimer analysis above.
Claims 10 and 20 further specify that when a first step is completed incorrectly or meets the threshold, the control unit communicates a notification before the operator starts a sequential second step, or the operator is prohibited from starting the second step until approval is communicated. Sequencing the steps and gating progression based on the compliance determination further describes the abstract idea of managing the maintenance quality control process.
Claim 11 further specifies that at least one of the plural sources includes sensors operably coupled with a wearable body. This limitation describes generic sensor hardware and does not recite any improvement to computer functionality or any other technical advance beyond the abstract idea itself.
Claim 12 further specifies that the control unit is an AI or machine-learning system. As discussed above, this amounts to mere instructions to implement the abstract idea on a computer. See MPEP § 2106.05(f).
Claim 18 further specifies that the control unit communicates with the operator during the maintenance event via the display device. Communicating with the operator via a display device is well-known extra-solution activity as addressed in the Berkheimer analysis above.
Claim 19 further specifies that the control unit determines that the operator's actions meet the correct action threshold and communicates a notification of approval. Determining compliance is part of the abstract idea, and communicating approval is well-known extra-solution activity as addressed in the Berkheimer analysis above.
Therefore, claims 1–20 are not drawn to eligible subject matter, as they are directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1–11 and 13–20 are rejected under 35 U.S.C. 103 as being unpatentable over Soldani et al. (US 10,049,111 B2) in view of Kathirvel et al. (US 10,065,750 B2), Pandya et al. (US 11,803,955 B1), and Worrall et al. (US 11,450,145 B2).
Claims 1 and 13: Soldani discloses a portable monitoring system and method (maintenance assistance system 100; FIG. 1; col. 3, ll. 45–67), comprising:
a control unit including one or more processors configured to access a technical specification associated with a maintenance event of an aircraft, the maintenance event including one or more steps configured to be completed by an operator to complete the maintenance event (FIG. 1; col. 4, ll. 1–9; col. 5, ll. 15–24. Soldani's display selection means 130 are "calculation means notably comprising a processor" that access a database storing maintenance assistance data associated with aircraft equipment, the maintenance assistance data comprising "instructions for adjusting or handling the equipment; [and] a list of elementary steps to be performed when a maintenance operation is to be carried out on this equipment, for example elementary steps to be performed when this equipment is to be replaced, moved, tested for correct function, etc.");
a communication system configured to establish wireless connections (col. 3, ll. 45–67; col. 4, ll. 1–9. The portable device 120 and the display selection means 130 "are in communication with one another, for example via a wireless communication."); and
the data being associated with ... characteristics of the aircraft during the one or more steps of the maintenance event (col. 4, ll. 35–43; col. 5, ll. 31–44. Camera 121 captures images of aircraft equipment during maintenance operations and transmits those images to the display selection means 130, and the shape recognition means 131 identify the equipment and determine an equipment identifier, the images depicting the configuration and state of the aircraft equipment, i.e., data associated with characteristics of the aircraft during the maintenance event. Soldani further discloses that the display selection means "perform a classification of the identifiers of interest on the basis of the corresponding maintenance operation, so that a user does not begin a new maintenance operation until after a maintenance operation previously begun has been completed" (col. 9, ll. 60–67; col. 10, ll. 1–30; col. 11, ll. 1–32), i.e., monitoring the maintenance event based on the captured data and the technical specifications.).
Soldani does not appear to explicitly disclose that the technical specification identify[s] completion thresholds indicating how each of the one or more steps is to be completed by the operator; that the communication system is configured to establish wireless connections with plural sources; that the data is received from each of the plural sources and is in the form of two or more different types of modalities; that the data is associated with actions taken by the operator; that the control unit is configured to analyze the two or more different types of modalities of the data from the plural sources to correlate at least some of the data associated with the actions taken by the operator with at least some of the data associated with the characteristics of the aircraft; or that the control unit is configured to examine the actions taken by the operator in real-time during each of the one or more steps of the maintenance event relative to the completion thresholds to determine that each of the one or more steps of the maintenance event meet the corresponding completion thresholds.
Kathirvel, however, discloses a communication system configured to establish wireless connections with plural sources (Abstract; col. 2, ll. 45–67; col. 4, ll. 45–55; col. 5, ll. 1–22. Kathirvel's wearable device 16 for managing aircraft maintenance operations comprises a wireless interface 58 configured to communicate with an onboard avionics system 18, an airline operations system 20, and an inventory management system 22. Kathirvel further discloses that the wearable device includes input devices 46 comprising an audio recording device and a video recording device (col. 4, ll. 20–28), from which speech data and video data are received during maintenance activities (col. 5, ll. 60–col. 6, ll. 15), and that aircraft information, maintenance information, and part information are received from the wirelessly connected computing systems 18–22 (col. 5, ll. 28–60).).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to incorporate the wearable multimodal sensing and plural-system wireless connectivity of Kathirvel into the AR-based aircraft maintenance system of Soldani. One would have been motivated to do this in order to provide hands-free multimodal data capture and direct wireless access to onboard avionics, airline operations, and inventory management data during maintenance, thereby improving the efficiency and information richness of the portable maintenance assistance system.
The Soldani/Kathirvel combination does not appear to explicitly disclose that the control unit receives data in the form of two or more different types of modalities from the plural sources.
Pandya, however, discloses the data being in the form of two or more different types of modalities received from the plural sources (FIG. 1; col. 5, ll. 25–67; col. 6, ll. 1–30. Pandya discloses multimodal sensory data collected from wirelessly networked plural sources including a computer vision system 110, a real-time locating system (RTLS) 120, a LIDAR system 130, and wearable sensors/mobile tag devices 105 worn by personnel, communicated over a local network to an edge computing device 140 for real-time monitoring and safety behavior compliance. The plural wirelessly connected sources provide data in different modalities — including image data from computer vision, positional data from RTLS, 3D point cloud data from LIDAR, and motion/orientation data from wearable sensors — to the edge computing device.).
Pandya further discloses the control unit configured to analyze the two or more different types of modalities of the data from the plural sources to correlate at least some of the data associated with the actions taken by the operator with at least some of the data associated with the characteristics of the aircraft (FIG. 1; col. 4, ll. 25–45. Pandya discloses that the edge computing device 140 receives a data stream including computer vision output data, RTLS location data, and LIDAR 3D point cloud data, and generates an "input feature dataset" by "aligning the data stream with respect to time and/or the identity of the object," i.e., correlating data associated with actions and position of personnel with data associated with characteristics of the environment and objects.).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to incorporate the wirelessly networked multimodal plural-source sensor architecture of Pandya into the aircraft maintenance system of Soldani/Kathirvel. One would have been motivated to do this in order to enable the control unit to receive data in multiple modalities from multiple wirelessly connected sensor sources during the maintenance event, and to correlate the multimodal data streams by aligning the data with respect to time and identity, thereby providing comprehensive real-time monitoring of both operator actions and aircraft equipment state.
Finally, the Soldani/Kathirvel/Pandya combination does not appear to explicitly disclose that the technical specification identify[s] completion thresholds indicating how each of the one or more steps is to be completed by the operator; or that the control unit is configured to examine the actions taken by the operator in real-time during each of the one or more steps of the maintenance event relative to the completion thresholds to determine that each of the one or more steps of the maintenance event meet the corresponding completion thresholds.
Worrall, however, discloses a system and method for monitoring procedure compliance (US 11,450,145 B2; Abstract; FIG. 6), comprising:
a technical specification identifying completion thresholds indicating how each of the one or more steps is to be completed (claim 1; col. 5, ll. 35–55. Worrall discloses that an operational process includes "a task or series of tasks that need to be performed to complete the operational process," and that the system determines "whether the user has complied with a compliance step, wherein the compliance step is to be completed prior to the performance of the task." The compliance step has defined requirements including performing the step "for a predetermined length of time," using "a particular object or consumable," and performing "different operations during completion of the compliance step" (col. 7, ll. 20–40), i.e., completion thresholds indicating how steps are to be completed.);
the data being associated with actions taken by the operator (claim 1; FIG. 6; col. 13, ll. 33–45; col. 14, ll. 1–10. Worrall's sensor 640, which may include one or more video cameras, senses "data pertaining to an interaction 630," and the "interaction 630 may involve actions or behaviors 632 of a human actor (e.g., a worker or technician being monitored by the system 600) as well as their use, movement, and/or operation of an identifiable object 634 during the behavior or action 632," such that the data is associated with both the operator's actions and characteristics of the object being worked on.); and
examine the actions taken by the operator in real-time during each of the one or more steps of the maintenance event relative to the completion thresholds to determine that each of the one or more steps of the maintenance event meet the corresponding completion thresholds (claim 1; FIG. 6; col. 13, ll. 25–35. Worrall discloses that the analytics platform performs "processing the data from the sensor during performance of the procedure to verify completion of the compliance step" (claim 1), using event timeline 628 that "controls which algorithms or object detections happen based on interactions," and "[w]hen all steps in the timeline 628 are successfully completed in the interaction 630, validation 670 is provided; otherwise, the feedback manager 624 is triggered.").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to incorporate the real-time procedure compliance monitoring, corrective feedback, and step-gating of Worrall into the multimodal aircraft maintenance monitoring system of Soldani/Kathirvel/Pandya. One would have been motivated to do this in order to convert the combined system from one that provides maintenance guidance and identifies parts into one that also verifies whether each maintenance step was actually performed correctly by the operator against defined procedural requirements, providing corrective feedback when a step does not meet the requirements and ensuring step completion before the operator proceeds, thereby improving maintenance quality, safety, and regulatory compliance.
Claim 2: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Kathirvel further discloses a system wherein the two or more different types of modalities includes one or more of video, text, audio, image, or touch (col. 4, ll. 20–28. Kathirvel's wearable device 16 includes input devices 46 comprising an audio recording device and a video recording device, i.e., audio and video modalities.).
The rationale for combining Kathirvel with Soldani is articulated above and is reincorporated herein by reference.
Claim 3: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a system wherein the control unit is configured to examine the two or more different types of modalities of the data to determine if the actions taken by the operator in completing the one or more steps are correct actions or incorrect actions based on the technical specification (claim 1; FIG. 6; col. 13, ll. 25–35; col. 7, ll. 40–65. Worrall's analytics platform processes sensor data during the procedure to verify completion of the compliance step, and determines whether "actions by the human actor in the interaction invalidate performance of the compliance step" (claim 1), i.e., determines whether the actions are correct or incorrect based on the procedure definition.).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 4: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Soldani further discloses a system further comprising an output device operably coupled with the control unit (FIG. 1; col. 5, ll. 45–67; col. 6, ll. 1–7. Soldani's augmented reality display means 122 receive data from the display selection means 130 and display images to the operator during maintenance.).
Worrall further discloses a system wherein the control unit is configured to communicate an alert to the operator during the maintenance event via the output device responsive to determining that the actions taken by the operator are incorrect actions (claim 1; col. 13, ll. 15–22; col. 7, ll. 50–65. Worrall teaches that "when the processing indicates actions by the human actor in the interaction invalidate performance of the compliance step," the system provides "real time feedback to the human actor regarding performance of the compliance step," the feedback comprising "that step was performed correctly or that it was not and should be performed again (sometimes with guidance on how to better comply with step requirements)" (col. 13, ll. 15–22).).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 5: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Soldani further discloses a system wherein the control unit is configured to communicate one or more instructions to assist the operator in completing the one or more steps of the maintenance event, the one or more instructions indicating the correct actions to be taken by the operator in completing the one or more steps of the maintenance event (col. 5, ll. 15–24; col. 5, ll. 45–67. Soldani's maintenance assistance data include "instructions for adjusting or handling the equipment" and "a list of elementary steps to be performed when a maintenance operation is to be carried out on this equipment," which are displayed in augmented reality over the equipment.).
Worrall further discloses a system wherein the feedback comprises instructions on repeating performance of the compliance step according to a set of requirements (claim 3; col. 8, ll. 5–25. Worrall teaches that feedback may include "notifying the user that he/she needs to wash his/her hands for an additional fifteen seconds" and may compare the user's performance with "an exemplary video of how the compliance step should be completed.").
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 6: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a system wherein the control unit is configured to confirm the completion of the one or more steps of the maintenance event (FIG. 6; col. 13, ll. 25–35. Worrall teaches that "[w]hen all steps in the timeline 628 are successfully completed in the interaction 630, validation 670 is provided," i.e., confirming completion of the steps.).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 7: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a system wherein the control unit is configured to identify that the actions taken by the operator during at least one of the one or more steps of the maintenance event did not meet the completion threshold of the at least one step (claim 1; col. 13, ll. 25–35; col. 7, ll. 40–55. Worrall teaches that the analytics platform processes sensor data during the procedure, and when the system "determines that the user has not complied with the compliance step at 104," the system identifies the non-compliance (col. 7, ll. 40–55).).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 8: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Soldani further discloses a system further comprising an output device operably coupled with the control unit (FIG. 1; col. 5, ll. 45–67. Augmented reality display means 122.).
Worrall further discloses a system wherein the control unit is configured to communicate a notification to the operator via the output device responsive to identifying that the actions taken by the operator did not meet the completion threshold of the at least one of the one or more steps (claim 1; col. 7, ll. 50–65; col. 13, ll. 15–22. Worrall teaches that when the user has not complied, "an embodiment may provide feedback to the user at 106," including "an audio notification (e.g., voice, buzzer, beep, etc.)" or "a visual notification (e.g., blinking light, video, pop-up display, etc.)" (col. 7, ll. 50–65).).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 9: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a system wherein the notification is configured to include a corrective action recommendation for the operator to complete in order to remedy the at least one of the one or more steps that was completed incorrectly (claim 3; col. 8, ll. 5–25; col. 13, ll. 15–22. Worrall teaches that the real time feedback comprises "an indication that the compliance step was completed improperly and instructions on repeating performance of the compliance step according to a set of requirements" (claim 3), and that the system may compare the user's video with "an exemplary video of how the compliance step should be completed" and "augment the video ... with the necessary feedback" (col. 8, ll. 15–25).).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 10: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a system wherein the at least one of the one or more steps that is completed incorrectly is a first step, wherein the control unit is configured to communicate the notification to the operator responsive to the operator completing the first step incorrectly and prior to the operator starting a sequential second step (claim 4; col. 7, ll. 40–50; col. 8, ll. 25–35. Worrall teaches that "when the system determines that the user has complied with the compliance step at 104, an embodiment may allow the user to progress to the next step of the operational process at 105," and that "[f]eedback may also include preventing the user from progressing to the next step of the operational process" (col. 8, ll. 25–35). Claim 4 further recites that "after the providing of the real time feedback," the system performs "processing of the sensor data captured by the sensor to verify completion of the compliance step and, in response, allowing the additional steps of the procedure to be performed by the human actor," i.e., the operator cannot proceed to a subsequent step until the prior step has been verified as properly completed.).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 11: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Kathirvel further discloses a system wherein at least one of the plural sources includes the one or more sensors operably coupled with a body, wherein the body is configured to be one or more of worn by or coupled to the operator during the maintenance event (col. 4, ll. 20–28; col. 2, ll. 45–58. Kathirvel's wearable device 16 includes input devices 46 comprising an audio recording device and a video recording device, the wearable device being eyeglasses worn by the user during aircraft maintenance.).
The rationale for combining Kathirvel with Soldani is articulated above and is reincorporated herein by reference.
Claim 14: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a method further comprising examining the two or more different types of modalities of the data to determine if the actions taken by the operator in completing the one or more steps are correct actions or incorrect actions based on the technical specification (claim 1; FIG. 6; col. 13, ll. 25–35; col. 7, ll. 40–65. Worrall's analytics platform processes sensor data during the procedure and determines whether the human actor's actions validate or invalidate performance of the compliance step.).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 15: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a method further comprising identifying that the actions taken by the operator in completing at least one of the one or more steps are incorrect actions (claim 1; col. 7, ll. 40–55. Worrall teaches that the system determines whether the user "has not complied with the compliance step at 104."); and
communicating a notification to the operator responsive to identifying that the actions taken by the operator in completing at least one of the one or more steps are incorrect actions (claim 1; col. 7, ll. 50–65; col. 13, ll. 15–22. Worrall teaches providing "real time feedback to the human actor" comprising "an indication that the compliance step was completed improperly and instructions on repeating performance of the compliance step according to a set of requirements" (claim 3).).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 16: Soldani discloses a wearable or portable monitoring system (maintenance assistance system 100; FIG. 1; col. 3, ll. 45–67), comprising:
a control unit including one or more processors operably coupled with the body, the control unit configured to wirelessly access one or more technical specifications associated with the maintenance or inspection event of the aircraft (FIG. 1; col. 4, ll. 1–9; col. 5, ll. 15–24. Display selection means 130, comprising "calculation means notably comprising a processor," in wireless communication with portable device 120 and configured to access maintenance assistance data from a database, the data comprising "a list of elementary steps to be performed when a maintenance operation is to be carried out on this equipment."); and
the data being associated with ... characteristics of the aircraft during the maintenance or inspection event (col. 4, ll. 35–43; col. 5, ll. 31–44. Camera 121 captures images of aircraft equipment during maintenance and transmits those images to the display selection means 130, which identify the equipment and determine an equipment identifier, the images depicting the configuration and state of the aircraft equipment.).
Soldani does not appear to explicitly disclose a body configured to be one or more of worn by or coupled to an operator; a display device; the one or more technical specifications identifying correct action thresholds associated with steps of the maintenance or inspection event; a communication system configured to establish wireless connections with plural sources; that the control unit is configured to receive data in two or more different types of modalities from the plural sources; that the data is associated with actions taken by the operator; that the control unit is configured to monitor the data associated with the actions taken by the operator in real-time ... relative to the correct action thresholds identified by the one or more technical specifications; or that the control unit is configured to communicate an alert via the display device responsive to determining that at least one of the actions taken by the operator in completing at least one of the steps does not meet the corresponding correct action threshold for the at least one step, the alert triggering a change in a display of the display device to include a corrective action recommendation for the operator to remedy the at least one of the steps that does not meet the corresponding correct action threshold.
Kathirvel, however, discloses a body configured to be one or more of worn by or coupled to an operator during a maintenance or inspection event of an aircraft (col. 2, ll. 45–58; col. 4, ll. 20–28. Kathirvel's wearable device 16 comprises eyeglasses worn by a user during aircraft maintenance operations.); a display device (col. 4, ll. 20–35. Kathirvel's wearable device 16 includes display system 24 integrated into the eyeglasses.); and a communication system configured to establish wireless connections with plural sources (col. 4, ll. 45–55; col. 5, ll. 1–22. Wireless interface 58 communicates with an onboard avionics system 18, an airline operations system 20, and an inventory management system 22. Kathirvel further discloses input devices 46 comprising an audio recording device and a video recording device (col. 4, ll. 20–28).).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to incorporate the wearable device form factor, display, plural-system wireless connectivity, and multimodal input devices of Kathirvel into the AR-based aircraft maintenance system of Soldani. One would have been motivated to do this in order to provide a hands-free wearable maintenance device with an integrated display and direct wireless access to multiple aircraft systems during maintenance operations.
The Soldani/Kathirvel combination does not appear to explicitly disclose that the control unit receives data in two or more different types of modalities from the plural sources.
Pandya, however, discloses data in two or more different types of modalities received from plural sources (FIG. 1; col. 5, ll. 25–67; col. 6, ll. 1–30. Pandya discloses multimodal sensory data collected from wirelessly networked plural sources including a computer vision system 110, a real-time locating system 120, a LIDAR system 130, and wearable sensors/mobile tag devices 105 worn by personnel, communicated over a local network to an edge computing device 140 for real-time monitoring and safety behavior compliance.).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to incorporate the wirelessly networked multimodal plural-source sensor architecture of Pandya into the aircraft maintenance system of Soldani/Kathirvel. One would have been motivated to do this in order to enable the control unit to receive data in multiple modalities from multiple wirelessly connected sensor sources during the maintenance event, providing comprehensive real-time monitoring coverage of both operator actions and aircraft equipment state.
Finally, the Soldani/Kathirvel/Pandya combination does not appear to explicitly disclose the one or more technical specifications identifying correct action thresholds associated with steps of the maintenance or inspection event; that the data is associated with actions taken by the operator; that the control unit is configured to monitor the data associated with the actions taken by the operator in real-time ... relative to the correct action thresholds; or that the control unit is configured to communicate an alert via the display device responsive to determining that at least one of the actions taken by the operator in completing at least one of the steps does not meet the corresponding correct action threshold ... the alert triggering a change in a display of the display device to include a corrective action recommendation.
Worrall, however, discloses the one or more technical specifications identifying correct action thresholds associated with steps of the maintenance or inspection event (claim 1; col. 5, ll. 35–55; col. 7, ll. 20–40. Worrall's analytics platform retrieves a procedure definition including "a compliance step to be performed prior to additional steps of the procedure" (claim 1), the compliance step subject to defined requirements including performing the step "for a predetermined length of time," using "a particular object or consumable," and performing "different operations during completion of the compliance step" (col. 7, ll. 20–40), i.e., correct action thresholds defining how steps are to be completed.);
the data being associated with actions taken by the operator (claim 1; FIG. 6; col. 13, ll. 33–45; col. 14, ll. 1–10. Worrall's sensor 640 senses "data pertaining to an interaction 630," and the "interaction 630 may involve actions or behaviors 632 of a human actor ... as well as their use, movement, and/or operation of an identifiable object 634," such that the data is associated with both the operator's actions and characteristics of the object being worked on.);
the control unit is configured to monitor the data associated with the actions taken by the operator in real-time ... relative to the correct action thresholds identified by the one or more technical specifications (claim 1; FIG. 6; col. 13, ll. 5–35. Worrall discloses that the analytics platform 610 performs "processing the data from the sensor during performance of the procedure to verify completion of the compliance step" (claim 1), using business rules 622 and event timeline 628 in real time (col. 13, ll. 25–35).); and
the control unit is configured to communicate an alert via the display device responsive to determining that at least one of the actions taken by the operator in completing at least one of the steps does not meet the corresponding correct action threshold for the at least one step, the alert triggering a change in a display of the display device to include a corrective action recommendation for the operator to remedy the at least one of the steps that does not meet the corresponding correct action threshold (claims 1, 3; col. 13, ll. 15–22; col. 7, ll. 50–65; col. 8, ll. 5–25. Worrall teaches that "when the processing indicates actions by the human actor in the interaction invalidate performance of the compliance step," the system provides "real time feedback to the human actor" (claim 1), the feedback comprising "an indication that the compliance step was completed improperly and instructions on repeating performance of the compliance step according to a set of requirements" (claim 3). Worrall further teaches that the system may provide "a visual notification (e.g., blinking light, video, pop-up display, etc.)" (col. 7, ll. 55–65) and may compare the user's performance with "an exemplary video of how the compliance step should be completed" and "augment the video ... with the necessary feedback" (col. 8, ll. 15–25).).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to incorporate the real-time procedure compliance monitoring, corrective feedback, and step-gating of Worrall into the multimodal wearable aircraft maintenance system of Soldani/Kathirvel/Pandya. One would have been motivated to do this in order to convert the combined system from one that provides maintenance guidance and identifies parts into one that also verifies whether each maintenance step was performed correctly by the operator against defined procedural requirements, providing corrective feedback through the wearable display when a step does not meet those requirements, thereby improving maintenance quality, safety, and regulatory compliance.
Claim 17: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Pandya further discloses a system wherein the control unit is configured to examine the actions taken by the operator during completion of the one or more steps of the maintenance or inspection event based on each of the two or more different types of modalities of the data (FIG. 1; col. 4, ll. 25–45; col. 5, ll. 25–67. Pandya discloses that the edge computing device 140 receives and processes data from each of the different modalities — computer vision output data, RTLS location data, and LIDAR 3D point cloud data — generating an input feature dataset that aligns the multimodal data streams, and processes the input feature dataset using a machine learning model to generate a safety-related result, i.e., examining actions based on each of the different modalities of data.).
The rationale for combining Pandya with Soldani/Kathirvel is articulated above and is reincorporated herein by reference.
Claim 18: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Soldani further discloses a system wherein the control unit is configured to communicate with the operator during the maintenance or inspection event via the display device (col. 5, ll. 45–67; col. 6, ll. 1–7. Soldani's augmented reality display means 122 display maintenance assistance data to the operator during the maintenance event.).
Claim 19: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a system wherein the control unit is configured to determine that the actions taken by the operator to complete at least one of the one or more steps of the maintenance or inspection event meet the corresponding correct action threshold identified by the one or more technical specifications (FIG. 6; col. 13, ll. 25–35; col. 7, ll. 40–50. Worrall teaches that "[w]hen all steps in the timeline 628 are successfully completed in the interaction 630, validation 670 is provided" (col. 13, ll. 30–32), and that "when the system determines that the user has complied with the compliance step at 104, an embodiment may allow the user to progress to the next step" (col. 7, ll. 40–45).); and
wherein the control unit is configured to communicate a notification of approval to the operator responsive to determining that the actions taken by the operator meet the correct action threshold (FIG. 6; col. 13, ll. 25–35. Worrall's validation 670 functions as a notification of approval communicated to the human actor upon successful completion of the compliance step. Worrall further teaches that "feedback may also be provided if the user has complied with all operations of the compliance step" (col. 8, ll. 40–45).).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 20: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above.
Worrall further discloses a system wherein the at least one of the one or more steps that meets the correct action threshold is a first step, wherein the operator is prohibited from starting a sequential second step until the control unit communicates the notification of approval of the first step to the operator (claim 4; col. 7, ll. 40–50; col. 8, ll. 25–35. Worrall teaches that "when the system determines that the user has complied with the compliance step at 104, an embodiment may allow the user to progress to the next step of the operational process at 105" (col. 7, ll. 40–50), and that "[f]eedback may also include preventing the user from progressing to the next step of the operational process" (col. 8, ll. 25–30). Claim 4 further recites "processing of the sensor data captured by the sensor to verify completion of the compliance step and, in response, allowing the additional steps of the procedure to be performed by the human actor," i.e., the operator cannot proceed until the prior step has been verified as properly completed.).
The rationale for combining Worrall with Soldani/Kathirvel/Pandya is articulated above and is reincorporated herein by reference.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Soldani et al. (US 10,049,111 B2) in view of Kathirvel et al. (US 10,065,750 B2), Pandya et al. (US 11,803,955 B1), and Worrall et al. (US 11,450,145 B2), and further in view of Cho (US 2022/0172594 A1).
Claim 12: The Soldani/Kathirvel/Pandya/Worrall combination discloses those limitations cited above, but does not appear to explicitly disclose a system wherein the control unit is an artificial intelligence (AI) or machine-learning system.
Cho, however, discloses a system wherein the control unit is an artificial intelligence (AI) or machine-learning system (¶¶ 93–106; FIGS. 2A–2B, 3A–3B. Cho uses a neural-network-based Long Short-Term Memory (LSTM) deep-learning model as the control logic to classify workers' motions from sensor data for real-time monitoring and alerts.).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to implement the control unit of the Soldani/Kathirvel/Pandya/Worrall combination as an AI or machine-learning system as taught by Cho. One would have been motivated to do this in order to improve the accuracy and robustness of recognizing and classifying the operator's actions from the multimodal sensor data for real-time maintenance compliance monitoring.
Other Relevant Prior Art
Schwartz et al. (20170210492), directed to an augmented reality system for assessing an affected area of an aircraft.
Afrasiabi et al. (10643329), directed to automated paint quality control for aircraft.
Lake et al. (8825276), directed to maintenance systems and methods for use in analyzing maintenance data.
Shin et al. (20250174135), directed to an aircraft turnaround monitoring system.
Hochman et al. (20220343292), directed to a system and method of documenting aircraft parts condition, repair, and overhaul.
Response to Arguments
Section 101:
Applicant argues that the claims are not directed to an abstract idea and should not be rejected under 35 U.S.C. § 101. Specifically, Applicant contends: (1) the amended claims recite limitations that cannot be performed by human activity or in the mind alone, such as establishing wireless connections between a communication system and plural sources; (2) it would be impractical for a human supervisor to manage and monitor the recited tasks in a reasonable amount of time; (3) the additional elements integrate the abstract idea into a practical application by confining it to a particular use that cannot be performed by a human; (4) the claims recite an improvement to aircraft maintenance monitoring systems over known single-mode monitoring approaches; and (5) the claims determine "how" each step is completed rather than merely "if" the step was completed, thereby imposing meaningful limits on the scope of the claims. These arguments are not persuasive.
Regarding Applicant's first argument — that a human supervisor cannot establish wireless connections between a communication system and plural sources — the Examiner respectfully notes that this limitation has been identified as a generic hardware additional element, not as part of the abstract idea. The abstract idea is the monitoring and evaluation of an operator's performance of maintenance procedures against defined requirements. The communication system configured to establish wireless connections with plural sources is a generic hardware component recited at a high level of generality that serves as the data-gathering infrastructure through which the abstract idea is implemented. The fact that a human cannot literally perform a hardware function does not transform the abstract idea into patent-eligible subject matter; rather, it confirms that the hardware is merely being used as a tool to perform the abstract idea. See MPEP § 2106.05(f); Alice Corp., 573 U.S. at 223.
Regarding Applicant's second argument — that it would be impractical for a human supervisor to correlate multimodal data and monitor the recited tasks in a reasonable amount of time — the Examiner respectfully notes that the relevant inquiry under Step 2A is not whether it would be practical or efficient for a human to perform the claimed evaluation at scale, but whether the claims recite a concept that is capable of being performed mentally or with pen and paper. See MPEP § 2106.04(a)(2)(III)(C) ("Claims can recite a mental process even if they are claimed as being performed on a computer."). A human supervisor consulting a maintenance manual, observing an operator performing maintenance steps, comparing the operator's actions against the procedural requirements, and determining whether each step was performed correctly is the same conceptual process recited in the claims, regardless of the speed or scale at which it is performed. That a computer can perform this evaluation faster or across more data types than a human does not change the character of the underlying abstract idea. See CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1375 (Fed. Cir. 2011) ("That purely mental processes can be unpatentable, even when performed by a computer, was confirmed by the Supreme Court.").
Regarding Applicant's third argument — that the additional elements integrate the abstract idea into a practical application because they cannot be performed by a human — the Examiner respectfully notes that the inquiry under Step 2A, Prong Two is whether the claim as a whole integrates the judicial exception into a practical application by, for example, improving the functioning of a computer or other technology, applying the judicial exception with a particular machine, or effecting a transformation of a particular article. See MPEP § 2106.04(d). The additional elements recited in the claims — a control unit with processors, a communication system, a wearable body, and a display device — are generic hardware components that serve as tools to implement the abstract idea. The recitation that the system uses wireless connections, plural sources, and multiple data modalities describes the data-gathering environment in which the abstract idea operates, but does not recite any specific technical improvement in how the data is sensed, transmitted, processed, or fused. Generally linking the abstract idea to a particular technological environment (e.g., wireless sensor networks in aircraft maintenance) does not integrate the abstract idea into a practical application. See MPEP § 2106.05(h).
Regarding Applicant's fourth argument — that the claims recite an improvement to aircraft maintenance monitoring systems, citing the specification's description of deficiencies in known single-mode monitoring approaches — the Examiner respectfully notes that the alleged improvement is to the maintenance monitoring process, not to the functioning of a computer or any particular technology. The claims recite using multiple data modalities to evaluate an operator's performance, but this describes an improvement in the quality of the evaluation (using more information to make a better determination), not a technical improvement in how the computer processes the data. An improvement to the abstract idea itself — here, a better way to evaluate maintenance compliance — does not constitute an improvement to technology or computer functionality for purposes of Step 2A, Prong Two. See SAP Am., Inc. v. InvestPic, LLC, 898 F.3d 1161, 1168 (Fed. Cir. 2018) ("What is needed is an inventive concept in the non-abstract application realm."); BSG Tech LLC v. BuySeasons, Inc., 899 F.3d 1281, 1290 (Fed. Cir. 2018) (using a particular type of data to improve decision-making does not provide an inventive concept).
Regarding Applicant's fifth argument — that the claims determine "how" each step is completed rather than merely "if" it was completed, and that this distinction imposes meaningful limits on the scope of the claims — the Examiner respectfully notes that whether the evaluation concerns how a step was performed or whether it was performed at all, the underlying concept remains the same: evaluating an operator's actions against procedural requirements and making a compliance determination. The "how vs. if" distinction refines the granularity of the evaluation but does not change the character of the abstract idea, add a technical improvement, or otherwise integrate the judicial exception into a practical application.
Accordingly, Applicant's arguments have been fully considered but are not persuasive. The rejection of claims 1–20 under 35 U.S.C. § 101 is therefore maintained.
Section 103:
Applicant's arguments with respect to claims 1–20 under 35 U.S.C. § 103 have been fully considered but are rendered moot in view of the new grounds of rejection set forth above, which were necessitated by Applicant's amendments to the claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER BUSCH whose telephone number is (571)270-7953. The examiner can normally be reached M-F 10-7.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Waseem Ashraf can be reached at 571-270-3948. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER C BUSCH/Examiner, Art Unit 3621
/WASEEM ASHRAF/Supervisory Patent Examiner, Art Unit 3621
Prosecution Insights