DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/10/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-5, 8-22 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
The indicated allowability of claim 10 is withdrawn in view of the newly discovered reference(s) Santarone et al. (US Publication Number 2019/0171780 A1). Rejections based on the newly cited reference(s) follow.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 10, 14-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sudharsan-Grober et al. (US Publication Number 2024/0168546 A1, hereinafter “Sudharsan”) and Santarone et al. (US Publication Number 2019/0171780 A1, hereinafter “Santarone”), further in view of Yaramasu (NPL, “Aircraft Electrical System Intermittent Arc Fault Detection and Location”, 2012).
(1) regarding claim 1:
As shown in fig. 2, Sudharsan disclosed a portable electronic device for assisting a user in locating a condition of a system within a real-world environment (para. [0073], note that FIG. 1 shows a method for identifying at least one a place of interest on a physical object through its 3D model in augmented reality view using AR device) comprising:
a display (fig. 4, para. [0097], note that [0097] FIG. 4 shows identifiers (shown as arrows 5) positioned around the places of interest as selected by the user. The POIs in this example are holes on a car door frame. At this stage, the display of virtual identifiers 5 (shown in FIG. 4));
an image capture device (para. [0123], note that the photo taking logic involves in its AR view a photo frame 9, as shown in FIG. 8); and
a processor configured to: receive, from the image capture device, image data associated with a physical representation of the real-world environment (para. [0075], note that in Step S9: a position of the physical object within the physical environment using a spatial awareness technology is determined);
align a three-dimensional virtual model with the real-world environment, wherein the three-dimensional virtual model is a virtual representation of the real-world environment (para. [0076], note that in Step S1: the 3D model with the physical object within the physical environment is aligned in an augmented reality view, [0077] Step S4: Adapting a transparency of the 3D model);
receive first information associated with the condition of the system (para. [0087], note that places of Interests (POI) are explained as any area or part of the machine, object, etc. in context, where the user needs to focus on, to perform a particular task);
determine a virtual location in the three-dimensional virtual model corresponding to the condition of the system (para. [0096], note that the point or small area of the intersection is instantiated as POI i.e. virtual location in the 3D model 2a and it is registered in the 3D map of the physical environment 1. Each of the POIs created this way thus will have its associated positional information).
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching wherein the system comprises an electrical wiring system in an installed state; receive second information associated with the electrical wiring system, wherein the second information includes a measurement of a distance from a starting point in the electrical wiring system to the condition; determine a virtual path along a length of at least one electrical wire of the electrical wiring system; and provide a graphical indicator to be displayed over the virtual representation of the real-world environment, wherein the graphical indicator is to be displayed at a display location on the display that corresponds to a physical location of the condition of the system in the real-world environment.
However, Santarone disclosed wherein the system comprises an electrical wiring system in an installed state (para. [0189], note that models of a Structure (including original models and As Built Models) may include routings of pipes, wires, conduits and other features of a Structure and the installed equipment within the Structure); determine a virtual path along a length of at least one electrical wire of the electrical wiring system (para. [0403], note that the orienteering function of the AVM relays a pathway 1711 via waypoint 1712 from the emergency responder's first position to a second position. The orienteering function directs the emergency responder on a pathway 1711 through a subject Structure around As Built emplaced elements 1713 or emergency-related obstacles); and provide a graphical indicator to be displayed over the virtual representation of the real-world environment, wherein the graphical indicator is to be displayed at a display location on the display that corresponds to a physical location of the condition of the system in the real-world environment (para. [0404], note that the pathway 1711 to an emergency responder's waypoint 1712 while responding to an emergency is displayed via augmented reality overlay through the subject Structure making use of As Built AVM data to direct an emergency responder around emplaced objects 1713 or emergency obstacles 1714 otherwise obstructing a given user's pathway 1711 to a waypoint 1712).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the system comprises an electrical wiring system in an installed state; determine a virtual path along a length of at least one electrical wire of the electrical wiring system; and provide a graphical indicator to be displayed over the virtual representation of the real-world environment, wherein the graphical indicator is to be displayed at a display location on the display that corresponds to a physical location of the condition of the system in the real-world environment. The suggestion/motivation for doing so would have been in order to assist augmented Virtual Model (AVM) in determining the nature of the emergency and a path to be taken by an emergency responder. By referencing the AVM on a smart device, such as one located in the visor of a firefighter, the emergency responder can quickly and easily locate a cause of the emergency and address it, as well as rescue lives endangered by the emergency (abs.). Therefore, it would have been obvious to combine Sudharsan with Santarone to obtain the invention as specified in claim 1.
In addition to that, Yaramasu disclosed receive second information associated with the electrical wiring system, wherein the second information includes a measurement of a distance from a starting point in the electrical wiring system to the condition (page 4, B. Modeling of a Load Circuit with Faulty Wire, para. [0003], note that representation is sufficient when the objective is to find the fault location. In Fig. 5, D1 denotes the distance of the fault location from the SSPC and D2 is that from the load end)).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the second information includes a measurement of a distance from a starting point in the electrical wiring system to the condition. The suggestion/motivation for doing so would have been in order to detect and locate fault in an aircraft system (Abs.). Therefore, it would have been obvious to combine Sudharsan and Santarone with Yaramasu to obtain the invention as specified in claim 7.
(2) regarding claim 2:
Sudharsan further disclosed the portable electronic device of claim 1, wherein the processor is further configured to generate the graphical indicator for display over the representation of the real-world environment (10, fig. 9, para. [0126], FIG. 9 shows taking a picture of previously identified place of interest on a physical object as an example where a technician marks something on a physical object 2b within a physical environment 1 as missing or wrong, especially errors, faults, scratches, dents, and wants to take a picture to protocol it).
(3) regarding claim 3:
Sudharsan further disclosed the portable electronic device of claim 1, wherein the graphical indicator comprises an icon, a symbol, text, or a combination thereof (10, fig. 9, note that an icon is displayed. Also see para. [0126]).
(4) regarding claim 14:
Sudharsan further disclosed the portable electronic device of claim 1, wherein determining of the alignment of the system in the real-world environment with the three-dimensional virtual model of the system further includes determining that at least one virtual object or fiducial of the three-dimensional virtual model is aligned with a corresponding physical object or fiducial in the real-world environment (para. [0086], note that [0086] FIG. 2 shows an alignment of a 3D model 2a on actual physical object in a physical environment 1. This step (step S1 in FIG. 1) may be necessary when the 3D model 2a needs to be aligned/overlaid in augmented reality onto an existing physical object 2b. Alignment is a common feature required when the overlaid 3D model along with its pre-imposed instructions or annotations are helping a service technician 3 to focus on any place of interest. In FIG. 2 the 3D model is arranged as an 3D outline), and wherein the processor is further configured to provide a notification when the system in the real-world environment is aligned with the three-dimensional virtual model of the system (para. [0086], note that alignment is a common feature required when the overlaid 3D model along with its pre-imposed instructions or annotations are helping a service technician 3 to focus on any place of interest. In FIG. 2 the 3D model is arranged as an 3D outline).
(5) regarding claim 15:
Sudharsan further disclosed the portable electronic device of claim 1, wherein the processor is further configured to: identify the system based on user input (para. [0078], note that the motion data is determined by tracking at least one gesture of a user related to the physical object); and receive second information including the three-dimensional virtual model of the system from a computing device (para. [0086], note that FIG. 2 shows an alignment of a 3D model 2a on actual physical object in a physical environment 1. This step (step S1 in FIG. 1) may be necessary when the 3D model 2a needs to be aligned/overlaid in augmented reality onto an existing physical object 2b. Alignment is a common feature required when the overlaid 3D model along with its pre-imposed instructions or annotations are helping a service technician 3 to focus on any place of interest).
(6) regarding claim 16:
Sudharsan further disclosed the portable electronic device of claim 1, wherein the processor is further configured to: provide a graphical representation of one or more components of the system for display on the display, wherein the one or more components are hidden from a view of the user (para. [0110], note that FIG. 6 shows touching and selecting an identifier 5 of a place of interest of a physical object 2b in a physical environment 1 by hand 3. This is especially usefully for selecting an identifier 5 to perform an inspection or to add inspection information, to mark something as missing or faulty is a common use case in inspections or quality assurance); and provide additional information associated with the one or more components for display on the display (para. [0115], note that FIG. 7 shows an example AR user manual 7 created according to the method step (S13, FIG. 1): Adding instructional data to said at least one place of interest. The AR user manual including instructional data 7 or work instructions. In almost all industry use cases mentioned above, there exists a need for the service technician to refer to a user manual 7 to review steps of instruction to complete a task).
Claim(s) 4-5, 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sudharsan and Santarone, and further in view of Yaramasu (NPL, “Aircraft Electrical System Intermittent Arc Fault Detection and Location”, 2012).
(1) regarding claim 4:
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching wherein the condition comprises a wiring fault or error of the system, and wherein the system comprising an electrical wiring system in an installed state.
However, Yaramasu teaches wherein the condition comprises a wiring fault or error of the system, and wherein the system comprising an electrical wiring system in an installed state (page 1, I. Introduction, para. [0001], note that the breached or degraded wire insulation can manifest as intermittent arc faults during harsh in-flight vibrations and weather conditions or under mechanical stresses on the wires).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the condition comprises a wiring fault or error of the system, and wherein the system comprising an electrical wiring system in an installed state. The suggestion/motivation for doing so would have been in order to detect and locate fault in an aircraft system (Abs.). Therefore, it would have been obvious to combine Sudharsan and Santarone with Yaramasu to obtain the invention as specified in claim 4.
(2) regarding claim 5:
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching wherein at least a portion of the electrical wiring system is hidden from a view of the user in the real-world environment.
However, Yaramasu teaches wherein at least a portion of the electrical wiring system is hidden from a view of the user in the real-world environment (Abs. note that Intermittent arc faults appear in aircraft power systems in unpredictable ways when the degraded wires are wet, vibrating against metal structures, or under mechanical stresses i.e. the wiring is inside an aircraft’s power system).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein at least a portion of the electrical wiring system is hidden from a view of the user in the real-world environment. The suggestion/motivation for doing so would have been in order to detect and locate fault in an aircraft system (Abs.). Therefore, it would have been obvious to combine Sudharsan and Santarone with Yaramasu to obtain the invention as specified in claim 5.
(3) regarding claim 8:
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching wherein the starting point is at one end of at least one electrical wire of the electrical wiring system.
However, Yaramasu teaches wherein the starting point is at one end of at least one electrical wire of the electrical wiring system (page 9, B. Results, para. [0001], note that the distance D1 to the fault is 19.8 m under fault condition 1, which is close to load, and D1 is 4.8 m under fault condition 2, which is near the power bus).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the information includes a measurement of a distance from a starting point in the electrical wiring system to the condition. The suggestion/motivation for doing so would have been in order to detect and locate fault in an aircraft system (Abs.). Therefore, it would have been obvious to combine Sudharsan and Santarone with Yaramasu to obtain the invention as specified in claim 8.
(4) regarding claim 9:
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching wherein the distance is determined by a fault locator, and wherein the fault locator comprises a time domain reflectometer.
However, Yaramasu teaches wherein the distance is determined by a fault locator, and wherein the fault locator comprises a time domain reflectometer (page 2, I. Introduction, para. [0005], note that Sequence time domain reflectometry (STDR) and spread spectrum time domain reflectometry (SSTDR) have been used to detect and locate intermittent arc faults).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the distance is determined by a fault locator, and wherein the fault locator comprises a time domain reflectometer. The suggestion/motivation for doing so would have been in order to detect and locate fault in an aircraft system (Abs.). Therefore, it would have been obvious to combine Sudharsan and Santarone with Yaramasu to obtain the invention as specified in claim 9.
Claim(s) 17-18, 20-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sudharsan-Grober et al. (US Publication Number 2024/0168546 A1, hereinafter “Sudharsan”) in view of Santarone et al. (US Publication Number 2019/0171780 A1, hereinafter “Santarone”).
(1) regarding claim 17:
As shown in fig. 2, Sudharsan disclosed a method for assisting a user in locating a condition of a system within a real-world environment (para. [0073], note that FIG. 1 shows a method for identifying at least one a place of interest on a physical object through its 3D model in augmented reality view using AR device) comprising:
a display (fig. 4, para. [0097], note that [0097] FIG. 4 shows identifiers (shown as arrows 5) positioned around the places of interest as selected by the user. The POIs in this example are holes on a car door frame. At this stage, the display of virtual identifiers 5 (shown in FIG. 4));
receiving, by one or more processors, image data associated with a physical representation of a real-world environment (para. [0075], note that in Step S9: a position of the physical object within the physical environment using a spatial awareness technology is determined);
aligning, by one or more processors, a three-dimensional virtual model with the real-world environment, wherein the three-dimensional virtual model is a virtual representation of the real-world environment (para. [0076], note that in Step S1: the 3D model with the physical object within the physical environment is aligned in an augmented reality view, [0077] Step S4: Adapting a transparency of the 3D model);
receiving, by the one or more processors, information associated with the condition of the system (para. [0087], note that places of Interests (POI) are explained as any area or part of the machine, object, etc. in context, where the user needs to focus on, to perform a particular task);
determining, by the one or more processors, a virtual location in the three-dimensional virtual model corresponding to the condition of the system (para. [0096], note that the point or small area of the intersection is instantiated as POI i.e. virtual location in the 3D model 2a and it is registered in the 3D map of the physical environment 1. Each of the POIs created this way thus will have its associated positional information).
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching wherein the system comprises an electrical wiring system in an installed state; determining a virtual path along a length of at least one electrical wire of the electrical wiring system; and providing a graphical indicator to be displayed over the virtual representation of the real-world environment, wherein the graphical indicator is to be displayed at a display location on the display that corresponds to a physical location of the condition of the system in the real-world environment.
However, Santarone disclosed wherein the system comprises an electrical wiring system in an installed state (para. [0189], note that models of a Structure (including original models and As Built Models) may include routings of pipes, wires, conduits and other features of a Structure and the installed equipment within the Structure); determining a virtual path along a length of at least one electrical wire of the electrical wiring system (para. [0403], note that the orienteering function of the AVM relays a pathway 1711 via waypoint 1712 from the emergency responder's first position to a second position. The orienteering function directs the emergency responder on a pathway 1711 through a subject Structure around As Built emplaced elements 1713 or emergency-related obstacles); and providing a graphical indicator to be displayed over the virtual representation of the real-world environment, wherein the graphical indicator is to be displayed at a display location on the display that corresponds to a physical location of the condition of the system in the real-world environment (para. [0404], note that the pathway 1711 to an emergency responder's waypoint 1712 while responding to an emergency is displayed via augmented reality overlay through the subject Structure making use of As Built AVM data to direct an emergency responder around emplaced objects 1713 or emergency obstacles 1714 otherwise obstructing a given user's pathway 1711 to a waypoint 1712).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the system comprises an electrical wiring system in an installed state; determining a virtual path along a length of at least one electrical wire of the electrical wiring system; and providing a graphical indicator to be displayed over the virtual representation of the real-world environment, wherein the graphical indicator is to be displayed at a display location on the display that corresponds to a physical location of the condition of the system in the real-world environment. The suggestion/motivation for doing so would have been in order to assist augmented Virtual Model (AVM) in determining the nature of the emergency and a path to be taken by an emergency responder. By referencing the AVM on a smart device, such as one located in the visor of a firefighter, the emergency responder can quickly and easily locate a cause of the emergency and address it, as well as rescue lives endangered by the emergency (abs.). Therefore, it would have been obvious to combine Sudharsan with Santarone to obtain the invention as specified in claim 17.
(2) regarding claim 18:
Sudharsan further disclosed the method of claim 17, wherein the graphical indicator comprises an icon, a symbol, text, or a combination thereof (10, fig. 9, note that an icon is displayed. Also see para. [0126]).
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching wherein the condition comprises a fault or error of the system.
However, Santarone disclosed wherein the condition comprises a fault or error of the system (para. [0079], note that vibration readings may also be used to quantify operation of appliances and equipment associated with the building, such as HVAC, circulators, water pumps, washers, dryers, refrigerators, dishwashers and the like. Vibration data may be analyzed to generate profiles for properly running equipment and equipment that may be faulty and/or failing, thus potentially leading to a Service Call or an emergency).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the condition comprises a fault or error of the system. The suggestion/motivation for doing so would have been in order to assist augmented Virtual Model (AVM) in determining the nature of the emergency and a path to be taken by an emergency responder. By referencing the AVM on a smart device, such as one located in the visor of a firefighter, the emergency responder can quickly and easily locate a cause of the emergency and address it, as well as rescue lives endangered by the emergency (abs.). Therefore, it would have been obvious to combine Sudharsan with Santarone to obtain the invention as specified in claim 18.
(2) regarding claim 21:
Sudharsan disclosed most of the subject matter as described as above except for specifically teaching the method further comprising: receiving second information associated with the electrical wiring system, wherein the second information includes a measurement of a distance from a starting point in the electrical wiring system to the condition, wherein the virtual path starts from a starting point and extends along the length of the at least one electrical wire until the distance is reached.
However, Santarone disclosed receiving second information associated with the electrical wiring system, wherein the second information includes a measurement of a distance from a starting point in the electrical wiring system to the condition (para. [0310], note that the geographic position may be calculated with a GPS reading from within the Structure. Measuring a distance to the at least three of the positioning reference devices may include, one or more of: relative signal strength received from wireless transmissions emanating from the at least three positioning reference devices), wherein the virtual path starts from a starting point and extends along the length of the at least one electrical wire until the distance is reached (para. [0403], note that the orienteering function of the AVM relays a pathway 1711 via waypoint 1712 from the emergency responder's first position to a second position. The orienteering function directs the emergency responder on a pathway 1711 through a subject Structure around As Built emplaced elements 1713 or emergency-related obstacles 1714, which may impede a more direct route to a waypoint 1712 during an emergency. Also see para. [0189], note that models of a Structure (including original models and As Built Models) may include routings of pipes, wires, conduits and other features of a Structure and the installed equipment within the Structure).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach receiving second information associated with the electrical wiring system, wherein the second information includes a measurement of a distance from a starting point in the electrical wiring system to the condition, wherein the virtual path starts from a starting point and extends along the length of the at least one electrical wire until the distance is reached. The suggestion/motivation for doing so would have been in order to assist augmented Virtual Model (AVM) in determining the nature of the emergency and a path to be taken by an emergency responder. By referencing the AVM on a smart device, such as one located in the visor of a firefighter, the emergency responder can quickly and easily locate a cause of the emergency and address it, as well as rescue lives endangered by the emergency (abs.). Therefore, it would have been obvious to combine Sudharsan with Santarone to obtain the invention as specified in claim 21.
The proposed rejection of claim 17, renders obvious the steps of the non-transitory computer-readable medium (para. [0070]) claim 20 because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claim 17 is equally applicable to claim 20.
Allowable Subject Matter
Claims 11-13 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: the prior arts made of record do not teach “wherein the distance is reached at a second virtual location of the electrical wiring system, wherein the second virtual location of the electrical wiring system corresponds to the virtual location of the condition, and wherein the processor is further configured to determine, based on the second virtual location, a second physical location of the electrical wiring system in the real-world environment that corresponds to the second virtual location of the electrical wiring system”, as claimed in claims 11, 19 and 22. Claims 12-13 depend on claim 11.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Liu et al. (NPL, “Real-Time FPGA-Based Hardware Neural Network for Fault Detection and Isolation in More Electric Aircraft”, 2019) disclosed an FPGA-based neural network method which includes two stages: off-line construction using TensorFlow and real-time monitoring on the FPGA.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Hilina K Demeter whose telephone number is (571) 270-1676.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon could be reached at (571) 270- 0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about PAIR system, see http://pari-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HILINA K DEMETER/Primary Examiner, Art Unit 2617
Prosecution Insights