Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-10, 12 and 13 are pending and have been examined on the merits set forth below.
Response to Arguments
Applicant’s arguments with respect to the rejections under 35 USC 101 have been fully considered and are persuasive. The rejection under 35 USC 101 has been withdrawn. The display of a captured image including an overlay that includes instruction information as annotations and having one or more user selectable icons included in the overlay configured to provide access to corresponding reference information integrates the abstract idea into a practical application.
Applicant’s arguments with respect to claim(s) rejected under 35 USC 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Examiner has introduced Brackney, US 2011/0115816. At least figures 2, 3 and 5 and paragraphs [0012, 0029, 0040] present a captured image on a device that includes an overlay including a selectable icon or button to retrieve additional information.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 7, 9, 10 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al, US 2015/0146007, in view of Ishikawa et al, US 2020/0042794, and Brackney, US 2011/0115816.
As per claim 1, Dusik et al discloses an information providing system for use in providing information of work carried out in a facility, the information providing system comprising: a working terminal; and an instructing terminal that communicates with the working terminal, (figure 1; [0020-0021] – processor communicates instruction information to the heads up display to show instructions for performing servicing of components) the working terminal comprising:
a working communication device that outputs imaging information including an image of the facility captured with a camera ([0020 display of image of component to be serviced); and
a working display that displays working information concerning the work carried out in the facility, the instructing terminal comprising:
an instructing display that displays the captured image based on the imaging information ([0020] – image includes indication of location of component to be serviced and [0021] image information of task completion);
an interface that receives instructing information serving to offer an instruction for the work carried out in the facility ([0020] – instructions for performing the service work); and
an instructing communication device that outputs the instructing information, the working terminal being configured to display, on the working display, the captured image processed based on the imaging information and the instructing information, as the working information (figure 3; [0011, 0012, 0020] – instructions and component information are displayed on the heads up display).
Dusik et al fails to explicitly disclose while Ishikawa et al discloses wherein processing of the captured image includes overlaying the instructing information as annotations on the captured image to highlight components of the facility to work ([0050, 0058-0059] – annotation of image related to facility tasks). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to annotate images as taught by Ishikawa et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
The combination of Dusik et al and Ishikawa et al fail to explicitly disclose the working information comprises captured image that has been processed having one or more user selectable icons included in the overlay, each selectable icon configured to provide access to corresponding reference information related to the work. Brackney, discloses a building operations tool that presents a captured image on a device that includes an overlay including a selectable icon or button to retrieve additional information. (At least figures 2, 3 and 5 and paragraphs [0012, 0029, 0040]).
It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to present data in the visual field using a translucent overlay of graphical elements including icons, as taught by Brackney since both relate to task assistance using an image display with overlay having user selectable icons and the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 2, Dusik et al discloses the information providing system according to claim 1, further comprising a server device that communicates with the working terminal and the instructing terminal, the server device comprising: a processor that generates the working information based on the imaging information and the instructing information; and a server communication device that outputs the working information to the working terminal ([0020], figure 3 – image is output along with instructions to perform the component servicing).
As per claim 3, Dusik et al discloses the information providing system according to claim 2, wherein the processor enlarges a part of the captured image included in the imaging information based on the instructing information and includes the captured image after being enlarged in the working information (figure 3 – enlarged portion of components that needs servicing).
As per claim 5, Dusik et al discloses the information providing system according to claim 2, wherein the server device includes a memory that stores reference information to be consulted for the work carried out in the facility, and the processor includes, in the working information, the reference information selected based on the instructing information ([0020] – instruction information can be accessed during servicing to assist with procedure).
As per claim 7, Dusik et al discloses the information providing system according to claim 2, wherein the server communication device further outputs the working information to the instructing terminal, and the instructing display displays the working information ([0020] – instruction information can be accessed during servicing to assist with procedure).
As per claim 9, Dusik et al discloses a server device for use in providing information of work carried out in a facility, the server device [0012] comprising: a server communication device that communicates with a working terminal and an instructing terminal (figure 1; [0020-0021] – processor communicates instruction information to the heads up display to show instructions for performing servicing of components); and
a processor [0003-0005, 0012], the server communication device obtaining, from the working terminal, imaging information including an image of the facility captured with a camera ([0020 display of image of component to be serviced);
obtaining, from the instructing terminal, instructing information serving to offer an instruction for work carried out in the facility, the processor processing the captured image based on the imaging information and the instructing information to generate working information concerning the work carried out in the facility that includes the captured image that has been processed ([0020] – instructions for performing the service work, image includes indication of location of component to be serviced and [0021] image information of task completion), and
the server communication device being configured to output the working information to the working terminal (figure 3; [0011, 0012, 0020] – instructions and component information are displayed on the heads up display).
Dusik et al fails to explicitly disclose while Ishikawa et al discloses wherein processing of the captured image includes overlaying the instructing information as annotations on the captured image to highlight components of the facility to work ([0050, 0058-0059] – annotation of image related to facility tasks). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to annotate images as taught by Ishikawa et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
The combination of Dusik et al and Ishikawa et al fail to explicitly disclose the working information comprises captured image that has been processed having one or more selectable icons overlaid thereon, each selectable icon configured to provide access to corresponding reference information related to the work. Brackney, discloses a building operations tool that presents a captured image on a device that includes an overlay including a selectable icon or button to retrieve additional information. (At least figures 2, 3 and 5 and paragraphs [0012, 0029, 0040]).
It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to present data in the visual field using a translucent overlay of graphical elements including icons, as taught by Brackney since both relate to task assistance using an image display with overlay having user selectable icons and the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 10, Dusik et al discloses an information providing method for use in providing information of work carried out in a facility, the information providing method comprising:
obtaining, from a working terminal, imaging information including an image of the facility captured with a camera ([0020 display of image of component to be serviced);
obtaining, from an instructing terminal, instructing information serving to offer an instruction for work carried out in the facility;
processing the captured image based on the imaging information and the instructing information to generate working information concerning the work carried out in the facility that includes the captured image that has been processed ([0020] – instructions for performing the service work, image includes indication of location of component to be serviced and [0021] image information of task completion); and
outputting the working information to the working terminal (figure 3; [0011, 0012, 0020] – instructions and component information are displayed on the heads up display).
Dusik et al fails to explicitly disclose while Ishikawa et al discloses wherein processing of the captured image includes overlaying the instructing information as annotations on the captured image to highlight components of the facility to work ([0050, 0058-0059] – annotation of image related to facility tasks). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to annotate images as taught by Ishikawa et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
The combination of Dusik et al and Ishikawa et al fail to explicitly disclose the working information comprises captured image that has been processed having one or more selectable icons overlaid thereon, each selectable icon configured to provide access to corresponding reference information related to the work. Brackney, discloses a building operations tool that presents a captured image on a device that includes an overlay including a selectable icon or button to retrieve additional information. (At least figures 2, 3 and 5 and paragraphs [0012, 0029, 0040]).
It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to present data in the visual field using a translucent overlay of graphical elements including icons, as taught by Brackney since both relate to task assistance using an image display with overlay having user selectable icons and the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 12, Dusik et al discloses the reference information includes a work procedure ([0020] – various ways to display the work to be performed).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al, Ishikawa et al and Brackney, as applied to claims 1 and 2 and further in view of Ordy et al, US 2016/0031570.
As per claim 4, Dusik et al discloses the information providing system according to claim 2, but fails to explicitly disclose wherein the processor translates a text included in the instructing information and includes the translated text in the working information. Ordy et al describes a maintenance system wherein instruction information is translated from one language to another [0055]. It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to translate instruction text as taught by Ordy et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al, Ishikawa et al and Brackney, as applied to claims 1, 2 and 5, in view of Rakshit et al, US 2021/0209440.
As per claim 6, Dusik et al discloses the information providing system according to claim 5, but fails to explicitly disclose wherein the working communication device outputs, to the server device, identification information used to identify a user of the working terminal, and the processor includes, in the working information, the reference information selected based on the instructing information and the identification information. Rakshit et al discloses a virtual assistant system that issues instructions to users based on their user profile [0084-0085]. It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al instructions based on a user as taught by Rakshit et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim(s) 8 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dusik et al, Ishikawa et al and Brackney, in view of Kodama et al, US 2014/0240484.
As per claim 8, Dusik et al discloses the information providing system according to claim 1, wherein the working terminal includes the camera [0021], but Dusik et al fails to explicitly disclose and the instructing communication device outputs, to the working terminal, remote control information for remote control of the camera. Kodama et al discloses a support system for performing a task wherein a head mount display is provided with a camera and wherein control information is output to the system to zoom or acquire a picture [0032-0035]. It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to send control operations to the camera as taught by Kodama et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 13, Dusik et al fails to explicitly disclose while Kodama et al discloses the reference information includes manuals of the facility ([0060-0061] – manual for procedure is displayed with images and descriptions). It would have been obvious to one of ordinary skill in the art at the time of the invention to include in the system of Dusik et al the ability to include manuals as taught by Kodama et al since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US Patent 10,735,691 to Schmirler et al discloses and industrial visualization system that generates and delivers virtual reality and augmented reality presentations of industrial facilities to wearable appliances. The presentation comprises 3D views of a facility with selectable icons and renders workflow presentations that guide uses through processes.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNA LOFTIS whose telephone number is (571)272-6736. The examiner can normally be reached M-F 7:00am-3:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached at 571-270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JOHNNA LOFTIS
Primary Examiner
Art Unit 3625
/JOHNNA R LOFTIS/Primary Examiner, Art Unit 3625
Prosecution Insights