DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements submitted on February 17, 2023 and November 26, 2024 have been considered by the Examiner.
Claim Objections
Claims 24-36 are objected to because of the following informalities. Appropriate correction is required.
Claim 24 recites “the displayed context of at least one of the one or more assets.” Claim 24 depends from claim 23, which recites “a displayed context in the one or more 3D models.” Accordingly, while claim 23 would provide antecedent basis for “the displayed context,” claim 23 does not provide antecedent basis for the more-specifically recited, “the displayed context of at least one of the one or more assets” like recited in claim 24.
Claims 25-29 depend from claim 24 and thereby include all of the limitations of claim 24. Accordingly, claims 25-29 are also objected to for the same reasons as noted above for claim 24.
In claim 30, the phrase “to execute a change the displayed context” is considered grammatically incorrect. Claims 31 and 32 depend from claim 30 and thereby include all of the limitations of claim 30, and are therefore objected to for the same reasons as noted for claim 30. Also, in claims 31 and 32, “the change the displayed context” is considered grammatically incorrect.
In claim 33, the phrase “to execute a change a displayed context” is considered grammatically incorrect. Claims 33-36 depend from claim 33 and thereby include all of the limitations of claim 33, and are therefore objected to for the same reasons as noted for claim 33. In addition, in claim 34, the phrase “the change the displayed context” is considered grammatically incorrect.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 21-28, 30-33 and 36 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Application Publication No. 2018/0130260 to Schmirler et al. (“Schmirler”).
Regarding claim 21, Schmirler describes a system that generates and delivers augmented reality (AR) or virtual reality (VR) presentations to a user via a wearable computer device, wherein the presentations can comprise three-dimensional (3D) holographic views of a plant facility or location within a plant facility (see e.g. paragraph 0050). Like claimed, Schmirler particularly discloses that the system comprises one or more computers comprising one or more processors and one or more non-transitory computer readable media, the one or more non-transitory computer readable media having program instructions stored thereon that when executed cause the one or more computers to (see e.g. paragraphs 0058-0059, 0066-0071, 0073 and 0075: Schmirler discloses that the system comprises one or more computing devices, each comprising one or more processors and memory, and wherein the memory comprises computer executable instructions that are executable to perform the following tasks):
generate, by the one or more processors, one or more 3D models in a 3D environment (see e.g. paragraphs 0073, 0075, 0085-0087, and FIG. 8: Schmirler discloses that the system can utilize one or more plant models to generate a three-dimensional virtual reality presentation of the areas within an industrial facility.);
enable, by the one or more processors, navigation of the one or more 3D models by an operator (see e.g. paragraphs 0091-0093 and 0100, and FIGS. 8-11: Schmirler discloses that the system tracks the location and orientation of a wearable device that presents the 3D virtual reality presentation of the industrial facility; as the user of the wearable device moves, the presentation by the wearable device is updated to enable navigation of the 3D virtual reality presentation of the industrial facility.);
display, by the one or more processors, 3D representations of one or more assets (see e.g. paragraph 0085 and FIG. 8: Schmirler discloses that the 3D virtual reality presentation can include 3D representations of industrial assets such as machines, conveyors, control cabinets and other industrial devices.); and
enable, by the one or more processors, a selection of the one or more assets in the 3D environment (see e.g. paragraph 0098 and FIG. 8: Schmirler discloses that a user can select one or more of the assets in the 3D virtual reality presentation by selecting an asset information icon rendered in association with each asset.).
Accordingly, Schmirler teaches a system like that of claim 21, which is for navigating one or more three-dimensional models.
As per claim 22, Schmirler further discloses that the one or more 3D models include at least a portion of an industrial facility (see e.g. paragraphs 0073 and 0085-0087, and FIG. 8). Accordingly, Schmirler further teaches a system like that of claim 22.
As per claim 23, Schmirler further discloses that the 3D environment adjusts automatically to alter a displayed context in the one or more 3D models (see e.g. paragraphs 0091-0093, 0101-0102 and 0106, and FIGS. 8-11: Schmirler discloses that the 3D environment is automatically adjusted as the user moves to alter a displayed context, e.g. view, of the one or more 3D models). Accordingly, Schmirler further teaches a system like that of claim 23.
As per claim 24, Schmirler further discloses that the displayed context of at least one of the one or more assets is altered based at least in part on at least one user selection of at least one of the one or more assets (see e.g. paragraphs 0098 and 0103: Schmirler discloses that a user can select an asset rendered in the 3D virtual reality presentation, e.g. by selecting an asset information icon rendered in association with the asset; in response, the displayed context of the asset is altered to present information relevant to the asset.). Accordingly, Schmirler further teaches a system like that of claim 24.
As per claim 25, Schmirler discloses that the displayed context includes contextual information (see e.g. paragraphs 0089, 0098 and 0101: Schmirler discloses that the displayed context includes contextual information such as production statistics and, in response to selecting an individual asset, information relevant to the asset.). Accordingly, Schmirler further teaches a system like that of claim 25.
As per claim 26, Schmirler teaches that the contextual information can include an alarm (e.g. fault) status of the one or more assets (see e.g. paragraphs 0098 and 0104). Accordingly, Schmirler further teaches a system like that of claim 26.
As per claim 27, Schmirler further teaches that the contextual information can include a maintenance record (e.g. a date on which maintenance was most recently performed, information regarding maintenance actions performed on the asset) for the one or more assets (see e.g. paragraph 0098). Accordingly, Schmirler further teaches a system like that of claim 27.
As per claim 28, Schmirler further teaches that the contextual information can include a diagram (e.g. schematics) for the one or more assets (see e.g. paragraph 0103 and 0110). Accordingly, Schmirler further teaches a system like that of claim 28.
As per claim 30, Schmirler further discloses that the system is configured to execute a change to the displayed context of the one or more assets as a reaction to a zoom-in command or a zoom-out command (see e.g. paragraphs 0100-0101 and FIG. 11: Schmirler teaches enabling a user to transition to a first-person view of the 3D virtual reality presentation of the industrial facility, whereby the transition comprises zooming-in to a selected target location, e.g. one or more assets, of the facility. The command for transitioning to the first-person view can thus be considered a zoom-in command, and results in executing a change to the displayed context, e.g. the view, of the one or more assets in the virtual reality presentation of the industrial facility.). Accordingly, Schmirler further teaches a system like that of claim 30.
As per claim 31, Schmirler further teaches that the change to the displayed context of the one or more assets for a zoom-in command (i.e. a command to transition to a first-person view) includes showing appropriate asset process graphics (e.g. production or machine statistics) (see e.g. paragraphs 0100-0102 and FIG. 11). Accordingly, Schmirler further teaches a system like that of claim 31.
As per claim 32, Schmirler further teaches that the change to the displayed context of the one or more assets for a zoom-in command (i.e. a command to transition to a first-person view) can include showing one or more alarms (see e.g. paragraphs 0100-0101 and 0104). Accordingly, Schmirler further teaches a system like that of claim 32.
As per claim 33, Schmirler further discloses that the system is configured to execute a change to the displayed context of the one or more assets as a reaction to a magnification of the one or more 3D models (see e.g. paragraphs 0100-0101 and 0181-0182, and FIG. 11: Schmirler teaches enabling a user to transition to a first-person view of the 3D virtual reality presentation of the industrial facility, whereby the transition comprises enlarging a selected area of the industrial facility including one or more assets therein, and changing a displayed context of the one or more assets. Because the one or more assets are enlarged by the transition to the first-person view, the transition can be considered a magnification of the one or more 3D models.). Accordingly, Schmirler further teaches a system like that of claim 33.
As per claim 36, Schmirler further demonstrates that the change to the displayed context is automatic (see e.g. paragraphs 0100-0101 and 0181-0182). Accordingly, Schmirler further teaches a system like that of claim 36.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 29 is rejected under 35 U.S.C. 103 as being unpatentable over the U.S. Patent Application Publication to Schmirler cited above, and also over U.S. Patent No. 9,915,929 to Dubinsky et al. (“Dubinsky”).
Regarding claim 29, Schmirler teaches a system like that of claim 25, as is described above, in which the displayed context of one or more assets is altered based at least in part on a user selection of the one or more assets, wherein the displayed context includes contextual information. Schmirler particularly discloses that the contextual information can include various information items of relevance to the one or more assets (see e.g. paragraph 0098). However, Schmirler does not explicitly disclose that the contextual information includes a standard operating procedure for the one or more assets, as is required by claim 29.
Dubinsky generally describes a platform for implementing a facility equipment monitoring service that facilitates walk-through inspections of equipment within respective facilities via mobile devices (see e.g. column 1, line 64 – column 2, line 8). Regarding the claimed invention, Dubinsky particularly teaches enabling a user of the mobile device to access contextual information for a selected piece of equipment within a facility, wherein the contextual information includes a standard operating procedure for the piece of equipment (see e.g. column 10, lines 4-29).
It would have been obvious to one of ordinary skill in the art, having the teachings of Schmirler and Dubinsky before the effective filing date of the claimed invention, to modify the contextual information taught by Schmirler, which is presented with respect to the one or more assets, so as to include a standard operating procedure like taught by Dubinsky for the one or more assets. It would have been advantageous to one of ordinary skill to present such a standard operating procedure, because it can be useful to a user operating or inspecting the one or more assets, as is suggested by Dubinsky (see e.g. column 10, lines 4-29). Accordingly, Schmirler and Dubinsky are considered to teach, to one of ordinary skill in the art, a system like that of claim 29.
Claims 34 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over the U.S. Patent Application Publication to Schmirler cited above, and also over U.S. Patent Application Publication No. 2011/0009984 to Mukhi et al. (“Mukhi”).
Regarding claim 34, Schmirler teaches a system like that of claim 33, as is described above, which is configured to execute a change to a displayed context of one or more assets as a reaction to a magnification of one or more 3D models. Schmirler particularly discloses that the change to the displayed context of the one or more assets can include a display of operational data regarding the one or more assets (see e.g. paragraphs 0100-0103 and FIG. 11). Schmirler, however, does not explicitly disclose that that change to the displayed context of the one or more assets includes supervisory control and data acquisition (SCADA) visualizations of operational data, as is required by claim 34.
Mukhi nevertheless generally teaches displaying operational data for one or more assets used in an industrial process via SCADA visualizations of the operational data (see e.g. paragraphs 0002-0004, 0014-0015 and 0031 and 0040-0041, and FIG. 3B).
It would have been obvious to one of ordinary skill in the art, having the teachings of Schmirler and Mukhi before the effective filing date of the claimed invention, to modify the operational data (i.e. the changed display context) taught by Schmirler so as to include SCADA visualizations of operational data like taught by Mukhi. It would have been advantageous to one of ordinary skill to present SCADA visualizations, because SCADA systems are commonly used to monitor process data, as is suggested by Mukhi (see e.g. paragraph 0002). Accordingly, Schmirler and Mukhi are considered to teach, to one of ordinary skill in the art, a system like that of claim 34.
As per claim 35, it would have been obvious, as is described above, to modify the operational data taught by Schmirler so as to include SCADA visualizations like taught by Mukhi. Mukhi suggests that the SCADA visualizations including substantially real-time visualizations of the operational data (see e.g. paragraphs 0004, 0014-0015 and 0019). Accordingly, the above-described combination of Schmirler and Mukhi is further considered to teach a system like that of claim 35.
Claims 37 and 38 are rejected under 35 U.S.C. 103 as being unpatentable over the U.S. Patent Application Publication to Schmirler cited above, and also over U.S. Patent No. 9,672,486 to Turpin (“Turpin”).
Regarding claim 37, Schmirler teaches a system like that of claim 22, as is described above, which is configured to generate one or more 3D models and display 3D representations of one or more assets, wherein the one or more 3D models include at least a portion of an industrial facility. Schmirler, however, does not explicitly teach displaying a map comprising location data of the one or more assets in the industrial facility, as is required by claim 37.
Turpin nevertheless generally teaches displaying a map comprising the location data of one or more assets in an industrial facility (see e.g. column 1, line 63 – column 2, line 6).
It would have been obvious to one of ordinary skill in the art, having the teachings of Schmirler and Turpin before the effective filing date of the claimed invention, to modify the system taught by Schmirler such that the one or more non-transitory computer readable media have further program instructions stored thereon that, when executed, cause the processors of the one or more computers to additionally display a map comprising location data of the one or more assets in the industrial facility, as is taught by Turpin. It would have been advantageous to one of ordinary skill to display such a map, because it would provide the user an overview of the assets within the facility, as is evident from Turpin (see e.g. column 1, line 63 – column 2, line 6). Accordingly, Schmirler and Turpin are considered to teach, to one of ordinary skill in the art, a system like that of claim 37.
As per claim 38, Schmirler further teaches that the system provides an ability to zoom-in to one or more areas of the industrial facility and/or individual assets in the 3D environment (see e.g. paragraph 0100). Accordingly, the above-described combination of Schmirler and Turpin is further considered to teach a system like that of claim 38.
Conclusion
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant’s disclosure. The applicant is required under 37 C.F.R. §1.111(C) to consider these references fully when responding to this action. In particular, the U.S. Patent Application Publication to Billi et al. cited therein teaches displaying a layered map presentation that can display, inter alia, the location of one or more assets in an industrial facility. The U.S. Patent Application Publication to Reshef cited therein describes a system for supervisory control and data acquisition (SCADA) that transmits data regarding a changed property to a user device for e.g. visualization thereon.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BLAINE T BASOM whose telephone number is (571)272-4044. The examiner can normally be reached Monday-Friday, 9:00 am - 5:30 pm, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached at (571)270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BTB/
1/10/2026
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141