Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on July 1, 2026, has been entered.
Response to Arguments
Applicant's arguments filed with respect to rejections under 35 USC 103 have been fully considered but they are not persuasive. Applicant’s arguments are centered on the new amendments to the claims. Examiner notes, the limitation just prior to the amendment describes the overlay that includes the instructing information as annotations highlight components of the facility to work and the last limitation describes the user selectable icons are included in the overlay. It is inherent to the claim that the icons are associated with a corresponding highlighted component since the claim already indicates the overlay that includes the instructing information as annotations highlight components of the facility to work.
Specifically, Applicant argues Brackney does not describe selectable icons corresponding to a highlighted component. Examiner points to figure 3 of Brackney. This clearly shows a selectable icon over component that highlights it as having an issue with flow and wherein the user can select the icon to gather more information. The rejection has been updated to reflect the amendments.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 7, 9, 10 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al, US 2015/0146007, in view of Brackney, US 2011/0115816.
As per claim 1, Dusik et al discloses an information providing system for use in providing information of work carried out in a facility, the information providing system comprising: a working terminal; and an instructing terminal that communicates with the working terminal, (figure 1; [0020-0021] – processor communicates instruction information to the heads up display to show instructions for performing servicing of components) the working terminal comprising:
a working communication device that outputs imaging information including an image of the facility captured with a camera ([0020 display of image of component to be serviced); and
a working display that displays working information concerning the work carried out in the facility, the instructing terminal comprising:
an instructing display that displays the captured image based on the imaging information ([0020] – image includes indication of location of component to be serviced and [0021] image information of task completion);
an interface that receives instructing information serving to offer an instruction for the work carried out in the facility ([0020] – instructions for performing the service work); and
an instructing communication device that outputs the instructing information, the working terminal being configured to display, on the working display, the captured image processed based on the imaging information and the instructing information, as the working information (figure 3; [0011, 0012, 0020] – instructions and component information are displayed on the heads up display).
Dusik et al fail to explicitly disclose processing of the captured image includes overlaying the instructing information as annotations on the captured image to highlight components of the facility to work and the working information comprises captured image that has been processed having one or more user selectable icons included in the overlay, each selectable icon being associated with a corresponding highlighted component and configured to provide access to corresponding reference information related to the work for the corresponding highlighted component. Brackney, discloses a building operations tool that presents a captured image on a device that includes an overlay including a selectable icon or button to retrieve additional information. (At least figures 2, 3 and 5 and paragraphs [0012, 0029, 0040]). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to present data in the visual field using a translucent overlay of graphical elements including icons, as taught by Brackney since both relate to task assistance using an image display with overlay having user selectable icons and the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 2, Dusik et al discloses the information providing system according to claim 1, further comprising a server device that communicates with the working terminal and the instructing terminal, the server device comprising: a processor that generates the working information based on the imaging information and the instructing information; and a server communication device that outputs the working information to the working terminal ([0020], figure 3 – image is output along with instructions to perform the component servicing).
As per claim 3, Dusik et al discloses the information providing system according to claim 2, wherein the processor enlarges a part of the captured image included in the imaging information based on the instructing information and includes the captured image after being enlarged in the working information (figure 3 – enlarged portion of components that needs servicing).
As per claim 5, Dusik et al discloses the information providing system according to claim 2, wherein the server device includes a memory that stores reference information to be consulted for the work carried out in the facility, and the processor includes, in the working information, the reference information selected based on the instructing information ([0020] – instruction information can be accessed during servicing to assist with procedure).
As per claim 7, Dusik et al discloses the information providing system according to claim 2, wherein the server communication device further outputs the working information to the instructing terminal, and the instructing display displays the working information ([0020] – instruction information can be accessed during servicing to assist with procedure).
As per claim 9, Dusik et al discloses a server device for use in providing information of work carried out in a facility, the server device [0012] comprising: a server communication device that communicates with a working terminal and an instructing terminal (figure 1; [0020-0021] – processor communicates instruction information to the heads up display to show instructions for performing servicing of components); and
a processor [0003-0005, 0012], the server communication device obtaining, from the working terminal, imaging information including an image of the facility captured with a camera ([0020 display of image of component to be serviced);
obtaining, from the instructing terminal, instructing information serving to offer an instruction for work carried out in the facility, the processor processing the captured image based on the imaging information and the instructing information to generate working information concerning the work carried out in the facility that includes the captured image that has been processed ([0020] – instructions for performing the service work, image includes indication of location of component to be serviced and [0021] image information of task completion), and
the server communication device being configured to output the working information to the working terminal (figure 3; [0011, 0012, 0020] – instructions and component information are displayed on the heads up display).
Dusik et al fail to explicitly disclose processing of the captured image includes overlaying the instructing information as annotations on the captured image to highlight components of the facility to work and the working information comprises captured image that has been processed having one or more user selectable icons included in the overlay, each selectable icon being associated with a corresponding highlighted component and configured to provide access to corresponding reference information related to the work for the corresponding highlighted component. Brackney, discloses a building operations tool that presents a captured image on a device that includes an overlay including a selectable icon or button to retrieve additional information. (At least figures 2, 3 and 5 and paragraphs [0012, 0029, 0040]). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to present data in the visual field using a translucent overlay of graphical elements including icons, as taught by Brackney since both relate to task assistance using an image display with overlay having user selectable icons and the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 10, Dusik et al discloses an information providing method for use in providing information of work carried out in a facility, the information providing method comprising:
obtaining, from a working terminal, imaging information including an image of the facility captured with a camera ([0020 display of image of component to be serviced);
obtaining, from an instructing terminal, instructing information serving to offer an instruction for work carried out in the facility;
processing the captured image based on the imaging information and the instructing information to generate working information concerning the work carried out in the facility that includes the captured image that has been processed ([0020] – instructions for performing the service work, image includes indication of location of component to be serviced and [0021] image information of task completion); and
outputting the working information to the working terminal (figure 3; [0011, 0012, 0020] – instructions and component information are displayed on the heads up display).
Dusik et al fail to explicitly disclose processing of the captured image includes overlaying the instructing information as annotations on the captured image to highlight components of the facility to work and the working information comprises captured image that has been processed having one or more user selectable icons included in the overlay, each selectable icon being associated with a corresponding highlighted component and configured to provide access to corresponding reference information related to the work for the corresponding highlighted component. Brackney, discloses a building operations tool that presents a captured image on a device that includes an overlay including a selectable icon or button to retrieve additional information. (At least figures 2, 3 and 5 and paragraphs [0012, 0029, 0040]). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to present data in the visual field using a translucent overlay of graphical elements including icons, as taught by Brackney since both relate to task assistance using an image display with overlay having user selectable icons and the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 12, Dusik et al discloses the reference information includes a work procedure ([0020] – various ways to display the work to be performed).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al and Brackney, as applied to claims 1 and 2 and further in view of Ordy et al, US 2016/0031570.
As per claim 4, Dusik et al discloses the information providing system according to claim 2, but fails to explicitly disclose wherein the processor translates a text included in the instructing information and includes the translated text in the working information. Ordy et al describes a maintenance system wherein instruction information is translated from one language to another [0055]. It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to translate instruction text as taught by Ordy et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al and Brackney, as applied to claims 1, 2 and 5, in view of Rakshit et al, US 2021/0209440.
As per claim 6, Dusik et al discloses the information providing system according to claim 5, but fails to explicitly disclose wherein the working communication device outputs, to the server device, identification information used to identify a user of the working terminal, and the processor includes, in the working information, the reference information selected based on the instructing information and the identification information. Rakshit et al discloses a virtual assistant system that issues instructions to users based on their user profile [0084-0085]. It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al instructions based on a user as taught by Rakshit et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim(s) 8 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al and Brackney, in view of Kodama et al, US 2014/0240484.
As per claim 8, Dusik et al discloses the information providing system according to claim 1, wherein the working terminal includes the camera [0021], but Dusik et al fails to explicitly disclose and the instructing communication device outputs, to the working terminal, remote control information for remote control of the camera. Kodama et al discloses a support system for performing a task wherein a head mount display is provided with a camera and wherein control information is output to the system to zoom or acquire a picture [0032-0035]. It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to send control operations to the camera as taught by Kodama et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 13, Dusik et al fails to explicitly disclose while Kodama et al discloses the reference information includes manuals of the facility ([0060-0061] – manual for procedure is displayed with images and descriptions). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to include manuals as taught by Kodama et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lairsey et al, US 2020/0252276 - System And Method For Hardware Management And Configuration In A Datacenter Using Augmented Reality And Available Sensor Data - information handling system may further determine an identity of the element based upon an identification message sent to the element via the wireless communication link, determine one of the slot locations based upon the identity, receive configuration information from the element via the wireless communication link, and display an augmented reality overlay on the display over the image data
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNA LOFTIS whose telephone number is (571)272-6736. The examiner can normally be reached M-F 7:00am-3:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached at 571-270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JOHNNA LOFTIS
Primary Examiner
Art Unit 3625
/JOHNNA R LOFTIS/Primary Examiner, Art Unit 3625
Prosecution Insights