Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are presented in the case.
Priority
Applicant's claim for the benefit of a prior-filed Provisional application 63/292,785 filed on 12/22/2021 is acknowledged.
Information Disclosure Statement
The information disclosure statements submitted on 05/20/2024, 09/09/2024, 02/14/2025 and 05/12/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Objections
Claim 17 is objected to because of the following informalities:
Claim 17, line 1 recites the phrase “the video feed” which should be “in the video feed”
For the informalities above and wherever else they may occur appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5, 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Geesey (US 20100042952 A1) in view of Ebner et al. (US 20150062118 A1) hereinafter Ebner.
As to independent claim 1, Geesey teaches a method for troubleshooting a piece of equipment, the method comprising: [assists in performing repairs and maintenance (troubleshooting) on components of an appliance (equipment) ¶21-25 "user interface provided by system 100, the user explores the inner working of the device and retrieves information describing a particular repair or maintenance activity"]
providing an equipment troubleshooting graphical user interface (GUI) to a user device; [interactive viewing screen ¶24-25, ¶29 "outputs data to an interactive viewing screen. The screen provides a user interface that allows a user to view the various components or subsystems of the device."]
providing, via the GUI, a 3D equipment model of a real piece of equipment, the 3D model comprising a plurality of model components each corresponding to a real component of the real piece of equipment; [provides model of real vehicles and appliances ¶47, ¶21-22 "stores models of automobiles and other vehicles"…"home appliances such as washing machines"…"(3D) models"]
associating each of the plurality of model components with one or more of the plurality of functions implemented using the corresponding real component; [data associating components and functions ¶38-39 "data includes text, video, graphics, or animations that describe how the component or subsystem operates and how the component or subsystem interacts with the rest of the vehicle. The data also describes the function of each component or subsystem and provides a description or illustration of the component or subsystem when it operates in good working order"]
Geesey does not specifically teach providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; receiving, from the user device, a selection of a function from the list; and updating the GUI to emphasize model components associated with the selected function.
However, Ebner teaches providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; [vehicle functions using lists/menus ¶21, ¶47 Fig. 5 " menu illustrated on the left-hand side, different driver assistance systems (FS) are offered for adjustment"]
receiving, from the user device, a selection of a function from the list; and [selects items such as driver assistance ¶47, ¶21 Fig. 5 FS2]
updating the GUI to emphasize model components associated with the selected function. [adapts image to emphasize steering wheel according with driver assistance function Fig. 5 19 ¶47 "The selected driver assistance system FS2 which is not indicated in a transparent manner but has a color border relates to the steering system of the motor vehicle 1, which is why the image 18 of the motor vehicle 1 has been adapted such that there is a top view from the front, the windshield appears transparent and the steering wheel 19 can be seen in a clearly emphasized manner"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey by incorporating the providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; receiving, from the user device, a selection of a function from the list; and updating the GUI to emphasize model components associated with the selected function disclosed by Ebner because both techniques address the same field of graphical user interfaces and by incorporating Ebner into Geesey better conveys the technical issues and provides improved context and support [Ebner ¶22].
As to dependent claim 2, the rejection of claim 1 is incorporated, Geesey and Ebner further teach wherein emphasizing a model component of the 3D model comprises one or more of brightening the model component, changing a color of the model component, enlarging the model component, or causing the model component to change from a hidden state to a visible state. [Ebner opacity, scaling, removing ¶47, ¶14 "In order to change the scene, provision can therefore be made for at least one of the at least one 3D object to be rotated and/or translated and/or scaled and/or adapted in terms of its opacity and/or for at least one 3D object to be added or removed"]
As to dependent claim 3, the rejection of claim 1 is incorporated, Geesey and Ebner further teach receiving, from the user device, a selection of a component of the 3D model; and [Ebner selects items such as driver assistance ¶47, ¶21 Fig. 5 FS2]
providing, via the GUI, additional information about the selected component. [Geesey text, video, animations ¶39 "after receiving the subsystem or component selection criteria, the system displays data for the selected subsystem or component in step 210. The data includes text, video, graphics, or animations that describe how the component or subsystem operates and how the component or subsystem interacts with the rest of the vehicle"]
As to dependent claim 5, the rejection of claim 1 is incorporated, Geesey and Ebner further teach receiving, from the user device, a command to hide a set of one or more model components; and [Geesey hidden fashion ¶41 and transparent/remove ¶43]
updating, in response to the command, the GUI such that the set of one or more model components is not visible. [Geesey updates render ¶42-43]
As to independent claim 13, Geesey teaches A user device for troubleshooting a piece of equipment, the user device comprising: [screen device that assists in performing repairs and maintenance (troubleshooting) on components of an appliance (equipment) ¶21-25 "user interface provided by system 100, the user explores the inner working of the device and retrieves information describing a particular repair or maintenance activity"]
a processing circuit comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: [processing unit, storage and software ¶26, ¶29]
providing an equipment troubleshooting graphical user interface (GUI) to a user device; [interactive viewing screen ¶24-25, ¶29 "outputs data to an interactive viewing screen. The screen provides a user interface that allows a user to view the various components or subsystems of the device."]
providing, via the GUI, a 3D equipment model of a real piece of equipment, the 3D model comprising a plurality of model components each corresponding to a real component of the real piece of equipment; [provides model of real vehicles and appliances ¶47, ¶21-22 "stores models of automobiles and other vehicles"…"home appliances such as washing machines"…"(3D) models"]
associating each of the plurality of model components with one or more of the plurality of functions implemented using the corresponding real component; [data associating components and functions ¶38-39 "data includes text, video, graphics, or animations that describe how the component or subsystem operates and how the component or subsystem interacts with the rest of the vehicle. The data also describes the function of each component or subsystem and provides a description or illustration of the component or subsystem when it operates in good working order"]
Geesey does not specifically teach providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; receiving, from the user device, a selection of a function from the list; and updating the GUI to emphasize model components associated with the selected function.
However, Ebner teaches providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; [vehicle functions using lists/menus ¶21, ¶47 Fig. 5 " menu illustrated on the left-hand side, different driver assistance systems (FS) are offered for adjustment"]
receiving, from the user device, a selection of a function from the list; and [selects items such as driver assistance ¶47, ¶21 Fig. 5 FS2]
updating the GUI to emphasize model components associated with the selected function. [adapts image to emphasize steering wheel according with driver assistance function Fig. 5 19 ¶47 "The selected driver assistance system FS2 which is not indicated in a transparent manner but has a color border relates to the steering system of the motor vehicle 1, which is why the image 18 of the motor vehicle 1 has been adapted such that there is a top view from the front, the windshield appears transparent and the steering wheel 19 can be seen in a clearly emphasized manner"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey by incorporating the providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; receiving, from the user device, a selection of a function from the list; and updating the GUI to emphasize model components associated with the selected function disclosed by Ebner because both techniques address the same field of graphical user interfaces and by incorporating Ebner into Geesey better conveys the technical issues and provides improved context and support [Ebner ¶22].
As to independent claim 18, Geesey teaches an equipment troubleshooting tool for providing 3D models of selected equipment, the equipment troubleshooting tool comprising [screen device that assists in performing repairs and maintenance (troubleshooting) on components of an appliance (equipment) ¶21-25 "user interface provided by system 100, the user explores the inner working of the device and retrieves information describing a particular repair or maintenance activity"]
a non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to implement operations comprising: [processing unit, storage and software ¶26, ¶29]
providing an equipment troubleshooting graphical user interface (GUI) to a user device; [interactive viewing screen ¶24-25, ¶29 "outputs data to an interactive viewing screen. The screen provides a user interface that allows a user to view the various components or subsystems of the device."]
providing, via the GUI, a 3D equipment model of a real piece of equipment, the 3D model comprising a plurality of model components each corresponding to a real component of the real piece of equipment; [provides model of real vehicles and appliances ¶47, ¶21-22 "stores models of automobiles and other vehicles"…"home appliances such as washing machines"…"(3D) models"]
associating each of the plurality of model components with one or more of the plurality of functions implemented using the corresponding real component; [data associating components and functions ¶38-39 "data includes text, video, graphics, or animations that describe how the component or subsystem operates and how the component or subsystem interacts with the rest of the vehicle. The data also describes the function of each component or subsystem and provides a description or illustration of the component or subsystem when it operates in good working order"]
Geesey does not specifically teach providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; receiving, from the user device, a selection of a function from the list; and updating the GUI to emphasize model components associated with the selected function.
However, Ebner teaches providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; [vehicle functions using lists/menus ¶21, ¶47 Fig. 5 " menu illustrated on the left-hand side, different driver assistance systems (FS) are offered for adjustment"]
receiving, from the user device, a selection of a function from the list; and [selects items such as driver assistance ¶47, ¶21 Fig. 5 FS2]
updating the GUI to emphasize model components associated with the selected function. [adapts image to emphasize steering wheel according with driver assistance function Fig. 5 19 ¶47 "The selected driver assistance system FS2 which is not indicated in a transparent manner but has a color border relates to the steering system of the motor vehicle 1, which is why the image 18 of the motor vehicle 1 has been adapted such that there is a top view from the front, the windshield appears transparent and the steering wheel 19 can be seen in a clearly emphasized manner"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey by incorporating the providing, via the GUI, a list of a plurality of functions performed by the real piece of equipment; receiving, from the user device, a selection of a function from the list; and updating the GUI to emphasize model components associated with the selected function disclosed by Ebner because both techniques address the same field of graphical user interfaces and by incorporating Ebner into Geesey better conveys the technical issues and provides improved context and support [Ebner ¶22].
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Geesey and Ebner, as applied in the rejection of claims 1 above, and further in view of Chakraborty et al. (US 20060167711 A1) hereinafter Chakraborty.
As to dependent claim 4, the rejection of claim 1 is incorporated.
Geesey and Ebner further teach receiving, from the user device, a selection of a component of the 3D model; [Geesey selection from user Fig. 2 208 ¶38 and shop ¶45].
Geesey and Ebner do not specifically teach providing, via the GUI, a link to add the selected component to a digital shopping cart in response to receiving the selection of the component; receiving, from the user device, a selection of the link; and in response to receiving the selection of the link, adding the selected component to the digital shopping cart.
However, Chakraborty teaches providing, via the GUI, a link to add the selected component to a digital shopping cart in response to receiving the selection of the component; [select items to add to a cart ¶113, hyperlinks ¶89]
receiving, from the user device, a selection of the link; and [interface with links in selection boxes ¶85]
in response to receiving the selection of the link, adding the selected component to the digital shopping cart. [selection boxes add to cart ¶85, ¶113]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the providing, via the GUI, a link to add the selected component to a digital shopping cart in response to receiving the selection of the component; receiving, from the user device, a selection of the link; and in response to receiving the selection of the link, adding the selected component to the digital shopping cart disclosed by Chakraborty because all techniques address the same field of graphical user interfaces and by incorporating Chakraborty into Geesey and Ebner [Chakraborty].
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Geesey and Ebner, as applied in the rejection of claims 1 above, and further in view of Golparvar-Fard et al. (US 20190325089 A1) hereinafter Golparvar-Fard.
As to dependent claim 6, the rejection of claim 1 is incorporated.
Geesey and Ebner do not specifically teach wherein providing the GUI further comprises providing a legend for identifying model components by category.
However, Golparvar-Fard teaches wherein providing the GUI further comprises providing a legend for identifying model components by category. [color legend of sections (category) ¶130 "a color legend corresponding to the color-coded sections."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the wherein providing the GUI further comprises providing a legend for identifying model components by category disclosed by Golparvar-Fard because all techniques address the same field of graphical user interfaces and by incorporating Golparvar-Fard into Geesey and Ebner provides more accurate information for improved decision making [Golparvar-Fard ¶4-5].
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Geesey and Ebner, as applied in the rejection of claims 1 above, and further in view of Sharma et al. (US 20110196655 A1) hereinafter Sharma.
As to dependent claim 7, the rejection of claim 1 is incorporated.
Geesey and Ebner do not specifically teach wherein the real components of the real piece of equipment are components of a hydraulic system.
However, Sharma teaches wherein the real components of the real piece of equipment are components of a hydraulic system. [information from hydraulic system ¶31-32]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the wherein the real components of the real piece of equipment are components of a hydraulic system disclosed by Sharma because all techniques address the same field of graphical user interfaces and by incorporating Sharma into Geesey and Ebner enables users to better interrogate 3d space to obtains more information [Sharma ¶39].
Claims 8-9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Geesey and Ebner, as applied in the rejection of claims 1 and 18 above, and further in view of Murrill et al. (US 4930598 A) hereinafter Murrill.
As to dependent claim 8, the rejection of claim 1 is incorporated.
Geesey and Ebner do not specifically teach wherein the selected function includes lifting a platform or boom of the piece of equipment.
However, Murrill teaches wherein the selected function includes lifting a platform or boom of the piece of equipment. [lift with control panel Col. 10 ln. 17-37]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the wherein the selected function includes lifting a platform or boom of the piece of equipment disclosed by Murrill because all techniques address the same field of equipment diagnosis and by incorporating Murrill into Geesey and Ebner improves development of real devices that reduce issues like lateral sway [Murrill col. 1 ln. 5-13].
As to dependent claim 9, the rejection of claim 8 is incorporated, Geesey, Ebner and Murrill further teach wherein the emphasized model components include a model of a hydraulic pump and a model of a hydraulic hose. [Murrill Pump and Hose Col. 12 ln. 38-54]
As to dependent claim 19, the rejection of claim 18 is incorporated.
Geesey and Ebner do not specifically teach wherein the selected function includes lifting a platform or boom of the piece of equipment, and wherein the emphasized model components include a model of a hydraulic pump and a model of a hydraulic hose.
However, Murrill teaches wherein the selected function includes lifting a platform or boom of the piece of equipment, and wherein the emphasized model components include a model of a hydraulic pump and a model of a hydraulic hose. [lift with control panel Col. 10 ln. 17-37], [Murrill Pump and Hose Col. 12 ln. 38-54]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the wherein the selected function includes lifting a platform or boom of the piece of equipment, and wherein the emphasized model components include a model of a hydraulic pump and a model of a hydraulic hose disclosed by Murrill because all techniques address the same field of equipment diagnosis and by incorporating Murrill into Geesey and Ebner improves development of real devices that reduce issues like lateral sway [Murrill col. 1 ln. 5-13].
Claims 10-11, 14-15, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Geesey and Ebner, as applied in the rejection of claims 1 above, and further in view of Parfenov et al. (US 20160328883 A1) hereinafter Parfenov.
As to dependent claim 10, the rejection of claim 1 is incorporated.
Geesey and Ebner do not specifically teach receiving a video feed including the real piece of equipment; displaying, via the GUI, the video feed; and overlaying the model over the real piece of equipment in the video feed, wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other model components.
However, Parfenov teaches receiving a video feed including the real piece of equipment; [captured via camera ¶47]
displaying, via the GUI, the video feed; and [displays interface of video Fig. 1 101 ¶47]
overlaying the model over the real piece of equipment in the video feed, wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other model components. [generates AR overlays on relevant only ¶85 "AR system may highlight, or otherwise display in computer graphics, only the specific information or elements of a device that are relevant to step #4"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the receiving a video feed including the real piece of equipment; displaying, via the GUI, the video feed; and overlaying the model over the real piece of equipment in the video feed, wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other model components disclosed by Parfenov because all techniques address the same field of graphical user interfaces and by incorporating Parfenov into Geesey and Ebner better diagnoses and solve problems through analysis and interactions [Parfenov ¶16].
As to dependent claim 11, the rejection of claim 10 is incorporated, Geesey, Ebner and Parfenov further teach adjusting a position of a model component associated with the selected function based on a position of the corresponding real piece of equipment detected in the video feed. [Parfenov real-time and superimposing and remains aligns ¶89, ¶20 "superimposed over the actual graphic so that a feature of the computer graphics aligns to the feature on the actual graphic and remains aligned "]
As to dependent claim 14, the rejection of claim 13 is incorporated.
Geesey and Ebner do not specifically teach receiving a video feed including the real piece of equipment; displaying, via the GUI, the video feed; and overlaying the model over the real piece of equipment in the video feed, wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other model components.
However, Parfenov teaches receiving a video feed including the real piece of equipment; [captured via camera ¶47]
displaying, via the GUI, the video feed; and [displays interface of video Fig. 1 101 ¶47]
overlaying the model over the real piece of equipment in the video feed, wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other model components. [generates AR overlays on relevant only ¶85 "AR system may highlight, or otherwise display in computer graphics, only the specific information or elements of a device that are relevant to step #4"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the receiving a video feed including the real piece of equipment; displaying, via the GUI, the video feed; and overlaying the model over the real piece of equipment in the video feed, wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other model components disclosed by Parfenov because all techniques address the same field of graphical user interfaces and by incorporating Parfenov into Geesey and Ebner better diagnoses and solve problems through analysis and interactions [Parfenov ¶16].
As to dependent claim 15, the rejection of claim 14 is incorporated, Geesey, Ebner and Parfenov further teach adjusting a position of a model component associated with the selected function based on a position of the corresponding real piece of equipment detected in the video feed. [Parfenov real-time and superimposing and remains aligns ¶89, ¶20 "superimposed over the actual graphic so that a feature of the computer graphics aligns to the feature on the actual graphic and remains aligned "]
As to dependent claim 17, the rejection of claim 14 is incorporated, Geesey, Ebner and Parfenov further teach detecting, based on shape of the real piece of equipment the video feed, a type of the real piece of equipment; and [identify shape attribute of device ¶62]
selecting, based on the detected type, the 3D model from a database comprising a plurality of 3D models. [selects a digital twin (DT) model from memory or network (database) ¶81, ¶83]
As to dependent claim 20, the rejection of claim 18 is incorporated.
Geesey and Ebner do not specifically teach wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other components of the 3D model.
However, Parfenov teaches wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other components of the 3D model. [generates AR overlays on relevant only ¶85 "AR system may highlight, or otherwise display in computer graphics, only the specific information or elements of a device that are relevant to step #4"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey and Ebner by incorporating the wherein updating the GUI to emphasize model components associated with the selected function comprises displaying the model components associated with the selected function and not displaying any other components of the 3D model disclosed by Parfenov because all techniques address the same field of graphical user interfaces and by incorporating Parfenov into Geesey and Ebner better diagnoses and solve problems through analysis and interactions [Parfenov ¶16].
Claims 12 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Geesey, Ebner and Parfenov, as applied in the rejection of claims 10 and 14 above, and further in view of Nguyen et al. (US 20180129276 A1) hereinafter Nguyen.
As to dependent claim 12, the rejection of claim 10 is incorporated.
Geesey, Ebner and Parfenov do not specifically teach detecting, based on the video feed, that a real component associated with the selected function is not present or out of position on the real piece of equipment; updating the GUI to emphasize a model component corresponding to the not present or out-of-position real component; and providing, via the GUI, instructions for installing the not present or out-of-position real component.
However, Nguyen teaches detecting, based on the video feed, that a real component associated with the selected function is not present or out of position on the real piece of equipment; [detects missing parts ¶122]
updating the GUI to emphasize a model component corresponding to the not present or out-of-position real component; and [displays information accordingly ¶123]
providing, via the GUI, instructions for installing the not present or out-of-position real component. [interface component with notification ¶123 out of positions (wrong place)¶108]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey, Ebner and Parfenov by incorporating the detecting, based on the video feed, that a real component associated with the selected function is not present or out of position on the real piece of equipment; updating the GUI to emphasize a model component corresponding to the not present or out-of-position real component; and providing, via the GUI, instructions for installing the not present or out-of-position real component disclosed by Nguyen because all techniques address the same field of graphical user interfaces and by incorporating Nguyen into Geesey, Ebner and Parfenov enhance user experience and productivity with performing maintenance [Nguyen ¶31-32].
As to dependent claim 16, the rejection of claim 10 is incorporated.
Geesey, Ebner and Parfenov do not specifically teach detecting, based on the video feed, that a real component associated with the selected function is not present or out of position on the real piece of equipment; updating the GUI to emphasize a model component corresponding to the not present or out-of-position real component; and providing, on the GUI, instructions for installing the not present or out-of-position real component.
However, Nguyen teaches detecting, based on the video feed, that a real component associated with the selected function is not present or out of position on the real piece of equipment; [detects missing parts ¶122]
updating the GUI to emphasize a model component corresponding to the not present or out-of-position real component; and [displays information accordingly ¶123]
providing, on the GUI, instructions for installing the not present or out-of-position real component. [interface component with notification ¶123 out of positions (wrong place)¶108]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the maintenance interface disclosed by Geesey, Ebner and Parfenov by incorporating the detecting, based on the video feed, that a real component associated with the selected function is not present or out of position on the real piece of equipment; updating the GUI to emphasize a model component corresponding to the not present or out-of-position real component; and providing, on the GUI, instructions for installing the not present or out-of-position real component disclosed by Nguyen because all techniques address the same field of graphical user interfaces and by incorporating Nguyen into Geesey, Ebner and Parfenov enhance user experience and productivity with performing maintenance [Nguyen ¶31-32].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
Browne et al. (US 20230154240 A1) teaches a model that defines locations of components and generate a digital twin (see ¶3).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Beau Spratt whose telephone number is 571 272 9919. The examiner can normally be reached 8:30am to 5:00pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at 571 272 7212. The fax phone number for the organization where this application or proceeding is assigned is 571 483 7388.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866 217 9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800 786 9199 (IN USA OR CANADA) or 571 272 1000.
/BEAU D SPRATT/Primary Examiner, Art Unit 2143
Prosecution Insights