DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/21/2025 is/are compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Office Action Summary
Claim(s) 1, 3, 5, 7, 12-17, 19, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bufi et al (US 2022/0366558 A1) in view of Lopez et al (US 2020/0026590 A1), and further in view of Yates et al (US 2023/0324900 A1).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bufi et al (US 2022/0366558 A1) in view of Lopez et al (US 2020/0026590 A1) and Yates et al (US 2023/0324900 A1), and further in view of Jones et al (US 2015/0019187 A1).
Claim(s) 6 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bufi et al (US 2022/0366558 A1) in view of Lopez et al (US 2020/0026590 A1) and Yates et al (US 2023/0324900 A1), and further in view of Lehmann et al (US 2014/0372348 A1).
Claim(s) 2, 9-11, 18, and 20 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3, 5, 7, 12-17, 19, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bufi et al (US 2022/0366558 A1) in view of Lopez et al (US 2020/0026590 A1), and further in view of Yates et al (US 2023/0324900 A1).
Regarding claim(s) 1, 19, and 21, Bufi teaches a computer program product comprising a non-transitory computer readable medium retaining program instructions, which instructions when read by a processor (Figure 1: Computing system 116; and Paragraph [0071]), cause the processor to perform:
receiving an image captured by a camera, the image depicting at least two parts of a monitored device, a first part of the at least two parts subject to at least one first failure mode, and a second part of the at least two parts subject to at least one second failure mode (Paragraph [0033] – Paragraph [0034]: “The image data may include a plurality of images acquired during rotation of the article […] The method may include acquiring image data of an article under inspection using a camera, providing the image data to a node computing device, analyzing the image data at the node computing device using a defect detection model trained to detect at least one defect type”);
identifying in the image the first part and the second part (Paragraph [0034]: “[…] object detection model configured to receive the image data as an input and generate defect data describing a detected defect as an output”);
verifying whether the first part complies with the at least one first failure mode or not, and verify whether the second part complies with the at least one second failure mode or not (Paragraph [0034]: “sending the defect data from the node computing device to a programmable logic controller (“PLC”) device, and determining at the PLC device whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data”); and
taking at least one action subject to the first part complying with the at least one first failure mode or the second part complying with the at least one second failure mode(Paragraph [0037]: “The method may further include generating a stop inspection command at the PLC device upon determining the defect data is unacceptable. The method may further include generating an alarm command at the PLC device upon determining the defect data is unacceptable and sending the alarm command to an alarm system configured to generate and output an alarm”).
Bufi fails to teaches to detecting whether the first part is assumed to comply with the at least one first failure mode, comprising using at least a first engine, and whether the second part is assumed to comply with the at least one second failure mode, comprising using at least a second engine; and taking at least one action subject to the first part complying with the at least one first failure mode or the second part complying with the at least one second failure mode, the at least one action aimed at avoiding a malfunction of the device.
However, Lopez teaches to detecting whether the first part is assumed to comply with the at least one first failure mode, comprising using at least a first engine, and whether the second part is assumed to comply with the at least one second failure mode, comprising using at least a second engine (Paragraph [0006]: “[…] The status of each component may be monitored by at least one sensor. The sensor may record numerical data to serve as a historical data record for the component. When a component fails, the sensor may note the failure as part of the historical data record. This may allow a user to determine exactly when the component failed and what caused the failure. The component may then be replaced and the product is able to continue working”; and Paragraph [0009] – Paragraph [0010]: “To predict failure, a model may need to be constructed for each component […] A separate model may be used for each component in a product […]”).
Bufi discloses a system that receives images captured by a camera and analyzes the images using an object detection model to generate defect data, and further verifies whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data, thereby determining whether a component complies with a failure condition. Lopez discloses determining failure of components using component-specific models, wherein a separate model may be constructed and used for each component to evaluate whether the component is associated with a failure mode.
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the system of Bufi to use the component specific models of Lopez to evaluate compliance of different components with respective failure modes, because Lopez teach that using separate models for individual components improves the accuracy and relevance of failure determination for each component. Such a modification merely involves applying known component specific failure modeling techniques to the image-based inspection system of Bufi to achieve predictable results, namely, determining whether each component complies with its associated failure mode based on image-derived defect data. This motivation for the combination of Bufi and Lopez is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Bufi and Lopez fails to teach to taking at least one action subject to the first part complying with the at least one first failure mode or the second part complying with the at least one second failure mode, the at least one action aimed at avoiding a malfunction of the device.
However, Yates teaches to taking at least one action subject to the first part complying with the at least one first failure mode or the second part complying with the at least one second failure mode, the at least one action aimed at avoiding a malfunction of the device (Paragraph [0082]: “method of implementing corrective action for various types of identified anomalies according to some embodiments”; and Paragraph [0092]: “In some embodiments, the system is configured to generate one or more actions based on a predicted failure. In some embodiments, the one or more actions include scheduling maintenance, rerouting electricity, ordering of replacement parts and/or assets, generating reports and/or initiating replacement of assets predicted to fail within a specified timeframe”).
Bufi discloses a system that receives images captured by a camera and analyzes the images using an object detection model to generate defect data, and further verifies whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data, thereby determining whether a component complies with a failure condition. Lopez further discloses determining failure of components using component-specific models, wherein a separate model may be constructed and used for each component to evaluate whether the component is associated with a failure mode. Yates discloses generating one or more actions based on a predicted failure of a component, including scheduling maintenance or performing corrective actions to prevent or mitigate malfunction of the system.
It would have been obvious to one of ordinary skill in the art to further modify the combined system of Bufi and Lopez to perform an action aimed at avoiding malfunction, as taught by Yates, because once a failure or non-compliance condition is determined for a component, taking a maintenance or corrective action to prevent malfunction is a well-understood and logical next step in component health monitoring systems. The combination of Bufi, Lopez, and Yates merely involves the predictable use of prior art elements according to their established functions image-based defect detection and verification, component specific failure evaluation, and failure-based maintenance actions. This motivation for the combination of Bufi, Lopez, and Yates is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 3, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Bufi teaches wherein the at least one first failure mode and the at least one second failure mode are static failure modes (Paragraph [0034]: “[…] object detection model configured to receive the image data as an input and generate defect data describing a detected defect as an output […] determining at the PLC device whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data”; and Paragraph [0211]: “The defect classification includes assigning a detected defect to a defect class. Defect classes are associated with particular defect types (e.g. dent, paint, scratch, etc.) the defect detection model 154 is trained to detect”).
Regarding claim(s) 5, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Bufi teach wherein the at least one first failure mode is a static failure mode (Paragraph [0034]: “[…] object detection model configured to receive the image data as an input and generate defect data describing a detected defect as an output […] determining at the PLC device whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data”) and where Lopez teaches the at least one second failure mode is a dynamic failure (read as “overheating failure”) mode (Paragraph [0030]: “The data extracted may include data indicating the time at which the component failed and what caused the component to fail. For example, in the case of an overheating failure, the extracted data may include the time at which the component stopped working and the temperature at which the component failed”).
Regarding claim(s) 7, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Lopez teaches wherein the at least one first failure mode and the at least one second failure mode are dynamic failure modes (Paragraph [0030]: “The data extracted may include data indicating the time at which the component failed and what caused the component to fail. For example, in the case of an overheating failure, the extracted data may include the time at which the component stopped working and the temperature at which the component failed”).
Regarding claim(s) 12, Bufi as modified by Lopez and Yates teaches the system according to claim 1 further comprising:
Where Bufi teaches a camera configured to capture the image of the monitored device (Paragraph [0033] – Paragraph [0034]: “The image data may include a plurality of images acquired during rotation of the article […] The method may include acquiring image data of an article under inspection using a camera, providing the image data to a node computing device, analyzing the image data at the node computing device using a defect detection model trained to detect at least one defect type”).
Regarding claim(s) 13, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Lopez teaches wherein the at least one action comprises alerting a user (Paragraph [0056]: “generating a health report at 448 may include generating a predicted type of failure for a particular component. Thus, the health report may serve to alert a user that a component is predicted to fail. Further, the health report may tell the user how the component is most likely to fail (e.g. overheating versus a mechanical failure)”).
Regarding claim(s) 14, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Yates teaches wherein the at least one action comprises scheduling a maintenance operation (Paragraph [0092]: “In some embodiments, the system is configured to generate one or more actions based on a predicted failure. In some embodiments, the one or more actions include scheduling maintenance, rerouting electricity, ordering of replacement parts and/or assets, generating reports and/or initiating replacement of assets predicted to fail within a specified timeframe”).
Regarding claim(s) 15, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Yates teaches wherein the at least one action comprises analyzing a trend of the failure mode (Paragraph [0092]: “In some embodiments, the system is configured to generate one or more actions based on a predicted failure. In some embodiments, the one or more actions include scheduling maintenance, rerouting electricity, ordering of replacement parts and/or assets, generating reports and/or initiating replacement of assets predicted to fail within a specified timeframe”).
Regarding claim(s) 16, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Bufi teaches wherein the at least one action comprises suggesting a change to the operation mode of the monitored device or a part thereof (Paragraph [0037]: “The method may further include generating a stop inspection command at the PLC device upon determining the defect data is unacceptable. The method may further include generating an alarm command at the PLC device upon determining the defect data is unacceptable and sending the alarm command to an alarm system configured to generate and output an alarm”).
Regarding claim(s) 17, Bufi as modified by Lopez and Yates teaches the system according to claim 1, where Bufi teaches wherein the at least one action comprises at least one item selected from the group consisting of:
changing a capture rate for the camera and setting an analysis rate for at least a portion of further images captured by the camera (Paragraph [0228]: “The frame rate refers to the frame rate of inference per single image. The frame rate may be determined by numerous factors (e.g. image size, number of layers, number of anchor boxes, algorithms within network, operations within network, floating point precision, layer fusion, etc.). A higher FPS inference by the node device 148 may decrease inspection time to levels more suitable to industrial manufacturing inspection processes”);
where Lopez teach sending a message to a person in charge (Paragraph [0056]: “generating a health report at 448 may include generating a predicted type of failure for a particular component. Thus, the health report may serve to alert a user that a component is predicted to fail. Further, the health report may tell the user how the component is most likely to fail (e.g. overheating versus a mechanical failure)”);
storing an alert in a storage device (Paragraph [0012]: “Although the following descriptions refer to a single processor and a single computer readable storage medium, the descriptions may also apply to a system with multiple processors and multiple computer readable storage mediums.”); and
updating a database (Paragraph [0011]: “Therefore, a model may be automatically updated or trained based on the results of the automatic evaluation, instead of being updated when a user is able to do an evaluation. This may allow for more accurate models to be generated without a user adding additional input data”).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bufi et al (US 2022/0366558 A1) in view of Lopez et al (US 2020/0026590 A1) and Yates et al (US 2023/0324900 A1), and further in view of Jones et al (US 2015/0019187 A1).
Regarding claim(s) 4, Bufi as modified by Lopez and Yates teaches the system according to Claim 3, but do not specifically teach wherein a static failure mode of the at least one first part causes the at least one second failure mode. However, Jones teach wherein a static failure mode of the at least one first part causes the at least one second failure mode (Figure 7; and Paragraph [0072]: “The cascading effect layout model may also illustrate linkages 706 (only one linkage being called out) between the nodes 702, which may illustrate how a failure of one system of the complex system may directly or indirectly result in failure of one or more other systems of the complex system”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time of the invention, to apply the failure propagation teachings of Jones to the multi-part monitoring and failure analysis system of Bufi, Lopez, and Yates, in order to more accurately model and assess system-level reliability. In complex monitored devices, it is well known that a failure in one part may cause or contribute to a failure in another part due to mechanical, functional, or operational interdependencies. Incorporating such causation analysis represents a predictable use of prior art elements according to their established functions. This motivation for the combination of Bufi, Lopez, Yates, and Jones is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Claim(s) 6 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bufi et al (US 2022/0366558 A1) in view of Lopez et al (US 2020/0026590 A1) and Yates et al (US 2023/0324900 A1), and further in view of Lehmann et al (US 2014/0372348 A1).
Regarding claim(s) 6, Bufi as modified by Lopez and Yates teaches the system according to Claim 5, where Bufi teaches wherein the processor (Figure 1: Computing system 116; and Paragraph [0071]) is adapted to analyze a static failure mode in the at least one first part (Paragraph [0034]: “[…] object detection model configured to receive the image data as an input and generate defect data describing a detected defect as an output […] determining at the PLC device whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data”; and Paragraph [0211]: “The defect classification includes assigning a detected defect to a defect class. Defect classes are associated with particular defect types (e.g. dent, paint, scratch, etc.) the defect detection model 154 is trained to detect”)
Bufi fails to teach motion of the at least one second part that does not comply with an expected motion (read as “desired route”). However, Lehmann teaches motion of the at least one second part that does not comply with an expected motion (Paragraph [0066]: “The present process provides a crowd model which models the behavior of the crowd based on fluid dynamics […] each sub-population having a characteristic behavior, such as a desired route or a stochastic behavioral pattern”; and Paragraph [0082] – Paragraph [0083]: “auxiliary low-order stochastic models and tolerances which governs the dynamics of temporal variations in parametric values of the crowd model coefficients which are determined to be expected, typical, or normal […] The present system identifies a potential anomaly by detecting an abrupt change in the parameters of the crowd model […] if the abrupt change invalidates the predictions of the auxiliary stochastic model […]”).
It would have been obvious to one of ordinary skill in the art, at the time of the invention, to combine the static, image-based failure analysis of Bufi with the dynamic behavior analysis of Lopez and Yates, and to further apply the expected versus observed motion analysis taught by Lehmann, in order to improve failure detection accuracy in a system monitoring multiple parts. Such a combination represents the predictable use of known techniques, static inspection for non-moving parts and motion-based anomaly detection for moving parts, according to their established functions. This motivation for the combination of Bufi, Lopez, Yates, and Jones is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 8, Bufi as modified by Lopez and Yates teaches the system according to Claim 7, but do not specifically teach wherein the processor is adapted to analyze motion of the at least one first part that does not comply with an expected motion for the first part, and to analyze motion of the at least one second part that does not comply with an expected motion of the second part. However, Lehmann teach wherein the processor is adapted to analyze motion of the at least one first part that does not comply with an expected motion for the first part, and to analyze motion of the at least one second part that does not comply with an expected motion of the second part (Paragraph [0066]: “The present process provides a crowd model which models the behavior of the crowd based on fluid dynamics […] each sub-population having a characteristic behavior, such as a desired route or a stochastic behavioral pattern”; and Paragraph [0082] – Paragraph [0083]: “auxiliary low-order stochastic models and tolerances which governs the dynamics of temporal variations in parametric values of the crowd model coefficients which are determined to be expected, typical, or normal […] The present system identifies a potential anomaly by detecting an abrupt change in the parameters of the crowd model […] if the abrupt change invalidates the predictions of the auxiliary stochastic model […]”).
It would have been obvious to one of ordinary skill in the art, at the time of the invention, to combine the static, image-based failure analysis of Bufi with the dynamic behavior analysis of Lopez and Yates, and to further apply the expected versus observed motion analysis taught by Lehmann, in order to improve failure detection accuracy in a system monitoring multiple parts. Such a combination represents the predictable use of known techniques, static inspection for non-moving parts and motion-based anomaly detection for moving parts, according to their established functions. This motivation for the combination of Bufi, Lopez, Yates, and Jones is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Allowable Subject Matter
Claim(s) 2, 9-11, 18, and 20 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Relevant Prior Art Directed to State of Art
Hofig (US 9,483,342 B2) are relevant prior art not applied in the rejection(s) above. Hofig discloses A method for supporting failure mode and effects analysis, the method comprising: storing a meta-model in a computer-readable storage medium, the meta-model comprising generic parts of a plurality of technical systems, generic failure modes, and associations between the generic parts and the generic failure modes, the associations indicating, for each generic part, one or more generic failure modes associated with the generic part, wherein each generic failure mode identifies a type of failure for a respective generic part; instantiating, with a processor, the generic parts and the generic failure modes to generate part instances and failure mode instances, respectively, the part instances and the failure mode instances specifying an individual technical system of the plurality of technical systems; storing the part instances and the failure mode instances in association with the plurality of technical systems; and associating each failure mode instance with a respective one of the part instances based on the associations between the generic failure modes and the generic parts stored in the meta-model, wherein at least one of the part instances is associated with more than one failure mode instance.
Bell et al (US 9,122,605 B2) are relevant prior art not applied in the rejection(s) above. Bell discloses A method for determining multiple simultaneous fault conditions occurring on complex systems comprising: providing a plurality of failure mode signatures, each failure signature indicative of a corresponding failure mode in a plurality of failure modes; receiving symptoms of a complex system from monitors; when the received symptoms are indicative of multiple simultaneous fault conditions, then: mapping each of the received symptoms to a position in an array to define a symptom signature; calculating an error code for each of the plurality of failure modes using the plurality of failure mode signatures and the symptom signature; determining a Hamming distance for each of the plurality of failure modes from the calculated error codes; identifying failure modes in the plurality of failure modes that have a lowest Hamming distance; grouping the identified failure modes with the lowest Hamming distance into a plurality of simultaneous fault conditions such that identified failure modes with the lowest Hamming distance and having a same error code are grouped together in a same fault condition; and assigning all remaining failure modes in the plurality of failure modes with other Hamming distances into one of a plurality simultaneous fault conditions.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONGBONG NAH whose telephone number is (571) 272-1361. The examiner can normally be reached M - F: 9:00 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL MISTRY can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONGBONG NAH/Examiner, Art Unit 2674