DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 5 is objected to because of the following informalities:
In claim 5, line 3, “a machine learning tool” should read “the machine learning tool” as the limitation has antecedence in line 2.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 2 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites the limitation "the dialysis device" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-4, 6-17, and 19 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1-2 and 5 of copending Application No. 19/067,519 (US PGPub 20250285546) in view of Adi (US PGPub 20230274659).
The claims are taught as mapped below:
Application 19/072,787
Application 19/067,519
Claim 1 except wherein the generated output comprises a representation of a virtual character or person
Claim 1
Claim 2
Claim 1
Claim 3 except a dialysis device; and wherein the generated output comprises a representation of a virtual character or person
Claim 1
Claim 4 except a dialysis device; and wherein the generated output comprises a representation of a virtual character or person
Claim 2
Claim 16
Claim 5
Claim 17: wherein the information includes monitored activities of the user
Claim 5
While the cited claims are not identical to the instant claims, the limitations are anticipated as the cited claims are more specific than the instant claims and therefore anticipate the broader claim limitations such as “determining an assistance scheme for providing assistance to the user for carrying out the determined task” of cited claim 1 anticipates “compute an assistance scheme for carrying out a task to be executed by a user”. Regarding claims 1, 3, and 4, the ’519 Application is silent on the teachings of a dialysis device; and wherein the generated output comprises a representation of a virtual character or person. However, the Adi reference provides a teaching for a system and method for dialysis training using augmented reality including a virtual avatar for guiding a user (Abstract; Paragraphs 0041, 0044, 0064). Regarding claims 6-17 and 19, the ’519 Application is silent on the teachings of the instant claims. However, Adi further provides teachings for the system and method including using a camera to capture images as input data to determine status and the environment of the user, select and generate the dialysis path and steps, and provide an augmented/mixed reality environment for the user to view wherein the display includes a virtual avatar providing guidance and feedback including pointing and demonstrating the steps and wherein the difficulty of the steps can be based on the determined training state/level of the user based on monitoring the performance of the user for the steps (Paragraphs 0041-0042, 0050, 0061, 0064-0065, 0070-0075). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the ‘519 Application with the teachings of Adi to include the medical device being a dialysis device, generating the output to include a virtual character or person, capturing images as input data to determine an environment and output content to guide the user, providing an augmented/mixed reality environment based on the captured images and display the virtual character/avatar to provide guidance and feedback including pointing and demonstrating steps. One of ordinary skill would make such modification in order to provide more comprehensive training and better engage the user to improve user performance.
This is a provisional nonstatutory double patenting rejection.
Claims 1-4 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1-2, 4, and 16 of copending Application No. 19/072,808 (US PGPub 20250285559) in view of Adi (US PGPub 20230274659).
The claims are taught as mapped below:
Application 19/072,787
Application 19/072,808
Claim 1 except wherein the generated output comprises a representation of a virtual character or person.
Claim 1
Claim 2
Claims 1 & 2
Claim 3 except wherein the generated output comprises a representation of a virtual character or person
Claims 1 & 2
Claim 4
Claims 4 & 16
While the cited claims are not identical to the instant claims, the limitations are anticipated as the cited claims are more specific than the instant claims and therefore anticipate the broader claim limitations such as “compute user instructions for guiding a user through the training task” of cited claim 1 anticipates “compute an assistance scheme for carrying out a task to be executed by a user”. Regarding claims 1 and 3, the ’808 Application is silent on the teachings of wherein the generated output comprises a representation of a virtual character or person, though cited claim 16 which depends from cited claim 4 and teaches computing a virtual character or person for providing the assistance. However, the Adi reference provides a teaching for a system and method for dialysis training using augmented reality including a virtual avatar for guiding a user (Abstract; Paragraphs 0041, 0044, 0064). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the ‘808 Application with the teachings of Adi to generating the output to include a virtual character or person. One of ordinary skill would make such modification in order to provide more comprehensive training and better engage the user to improve user performance.
This is a provisional nonstatutory double patenting rejection.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-4, 6-17, and 19 is/are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Adi (US PGPub 20230274659).
Regarding claim 1, Adi teaches an augmented reality (AR) system (Paragraphs 0044, 0064-0065; “augmented reality (AR) goggles”), comprising: an AR device (Paragraphs 0044, 0064-0065; “augmented reality (AR) goggles”); and a control device (Paragraphs 0045-0046 teach a host machine for interfacing and controlling the system and performing the functions of the system) configured to: compute an assistance scheme for carrying out a task to be executed by a user (Paragraphs 0040, 0062, 0069, 0071 teach the system can set or select a dialysis path (assistance scheme) for a user); and generate an output via the AR device by applying the computed assistance scheme, wherein the generated output comprises a representation of a virtual character or person (Paragraphs 0041, 0044, 0064, 0066, 0070 teach the system includes a virtual avatar which guides the user through the dialysis procedure/path wherein the avatar is displayed via the output device such as AR goggles).
Regarding claim 2, Adi further teaches further comprising: a sensor configured to capture data related to the dialysis device and/or data related to an operation of the user (Paragraphs 0041-0042, 0058 teach the system includes an imaging input/camera (sensor) for capturing images of the user and the dialysis device).
Regarding claim 3, Adi teaches a medical system (Paragraphs 0005, 0039, 0055, 0077; “system” and “a computer system to execute a sequence of machine-readable instructions”), comprising: a dialysis device (Paragraphs 0041-0042; “dialysis machine”); an AR device (Paragraphs 0044, 0064-0065; “augmented reality (AR) goggles”); and a control device (Paragraphs 0045-0046 teach a host machine for interfacing and controlling the system and performing the functions of the system) configured to: compute an assistance scheme for carrying out a task to be executed by a user (Paragraphs 0040, 0062, 0069, 0071 teach the system can set or select a dialysis path (assistance scheme) for a user); and generate an output via the AR device by applying the computed assistance scheme, wherein the generated output comprises a representation of a virtual character or person (Paragraphs 0041, 0044, 0064, 0066, 0070 teach the system includes a virtual avatar which guides the user through the dialysis procedure/path wherein the avatar is displayed via the output device such as AR goggles).
Regarding claim 4, Adi teaches a method (Paragraphs 0006, 0074; “method”) for controlling an augmented reality (AR) device (Paragraphs 0044, 0064-0065; “augmented reality (AR) goggles”) for operating a dialysis device (Paragraphs 0041-0042; “dialysis machine”), the method comprising: computing, by an AR system, an assistance scheme for carrying out a task to be executed by a user (Paragraphs 0040, 0062, 0069, 0071 teach the system can set or select a dialysis path (assistance scheme) for a user); and generating, by the AR system, an output via the AR device by applying the computed assistance scheme, wherein the generated output comprises a representation of a virtual character or person (Paragraphs 0041, 0044, 0064, 0066, 0070 teach the system includes a virtual avatar which guides the user through the dialysis procedure/path wherein the avatar is displayed via the output device such as AR goggles).
Regarding claim 6, Adi further teaches further comprising: receiving input data from the dialysis device (Paragraphs 0041-0042, 0050, 0061 teach the system can read or interpret the control panel of the dialysis device and determine a status of the dialysis device); wherein computing the assistance scheme is based on the received input data (Paragraphs 0041-0043 teach the dialysis path can be based in part on the captured images including the images of the control panel).
Regarding claim 7, Adi further teaches wherein the generated output comprises an optical and/or acoustic output (Paragraphs 0040, 0064, 0070 teach the user can be provided guidance via the display (optical) of the virtual avatar, text boxes (optical) and/or audio such as spoken instructions).
Regarding claim 8, Adi further teaches wherein the computed assistance scheme comprises rules for representation of limb actuations of the virtual character or person (Paragraphs 0066, 0070, 0073, 0075 teach the virtual avatar/coach can include movement and positioning of the avatar and can point at objects as part of the guidance wherein “pointing at objects” would include actuation of the limb to point and the gestures are the “rules” as paragraph 0046 of the instant application defines the rules as movement of a limb to represent a desired movement as assistance to guide the user).
Regarding claim 9, Adi further teaches wherein, in the generated output, limbs of the virtual character or person are actuated (Paragraphs 0066, 0070, 0073, 0075 teach the virtual avatar/coach can include movement and positioning of the avatar and can point at objects as part of the guidance wherein “pointing at objects” would include actuation of the limb to point).
Regarding claim 10, Adi further teaches wherein the limb actuations of the virtual character or person represent or indicate a movement that is required to operate the dialysis device (Paragraphs 0066, 0070, 0072, 0073, 0075 teach the virtual avatar/coach can include movement and positioning of the avatar to guide the user such as pointing at an accessory to use or how to position the device and equipment).
Regarding claim 11, Adi further teaches wherein a level of detail for generating the output is adjustable (Paragraphs 0008, 0064-0065, 0071 teach the system can select the dialysis path, determine the instructional information presented to the user, and select a hint trigger (level of detail) based on a user training state/training level).
Regarding claim 12, Adi further teaches further comprising: monitoring activities of the user for determining a skill level of the user (Paragraphs 0008, 0042, 0064-0065, 0071 teach the system can track a training level of the user as a user training state based on the user’s performance such as number of attempts at a current path or timing out (monitored activities)).
Regarding claim 13, Adi further teaches wherein the level of detail for generating the output is adjusted based on the determined skill level of the user (Paragraphs 0008, 0064-0065, 0071 teach the system can select the dialysis path, determine the instructional information presented to the user, and select a hint trigger (level of detail) based on a user training state/training level including a determined training state for a new user).
Regarding claim 14, Adi further teaches further comprising: receiving data related to a skill level of the user of the user from a database (Paragraphs 0005, 0042, 0045, 0071 teach the system can store the dialysis process state in a memory wherein the dialysis process state includes the user training state which the system can update and thereby store in the memory such that the user training state (skill level) can be accessed from the memory (database)).
Regarding claim 15, Adi further teaches wherein the level of detail for generating the output is adjusted based on the received skill level and/or preferences of the user (Paragraphs 0008, 0064-0065, 0071 teach the system can select the dialysis path, determine the instructional information presented to the user, and select a hint trigger (level of detail) based on a user training state/training level).
Regarding claim 16, Adi further teaches wherein the assistance scheme and/or the output is computed based on information received from the AR device (Paragraphs 0041-0042, 0058, 0061, 0064-0066 teach the captured images (information received) can be used to determine the dialysis path and steps and provide guidance as part of the user’s augmented environment (output) based on the captured images).
Regarding claim 17, Adi further teaches wherein the information includes monitored activities of the user (Paragraphs 0037, 0071-0072 teach the system can track and monitor user performance including tracking the user’s body wherein the system can determine if the user successfully completed each step of the process) and/or time information (Paragraphs 0063, 0069 teach the system can include a training timer tracking the time of a current dialysis step and including a timeout action if the user exceeds the training timer).
Regarding claim 19, Adi further teaches further comprising: scanning, by at least one sensor, a spatial environment of the dialysis device; wherein the assistance scheme is computed based on data of the scanned spatial environment and/or the output is generated based on data of the scanned spatial environment (Paragraphs 0037-0038, 0041, 0050, 0064-0066 teach the system includes a camera (sensor) for capturing images of the user, the dialysis machine, and the user’s environment (spatial environment) such that the guidance (output) can include virtual and physical objects in an augmented environment of the user and wherein the dialysis path and steps can be based on the captured images).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adi in view of Buras et al. (US PGPub 20210327303), hereinafter referred to as Buras.
Regarding claim 5, Adi further teaches wherein the assistance scheme is computed by applying a machine learning tool; and/or wherein the output is generated by applying a machine learning tool. However, Buras teaches an augmented reality guidance system for providing procedure instruction data to guide medical equipment users wherein the guidance and feedback, including virtual images, video, and feedback, is generated by a machine learning model (Abstract; Paragraphs 0011, 0014, 0086, 0095, 0101, 0103-0104, 0133, 0138).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Adi to incorporate the teachings of Buras by applying the technique of using a machine learning model to generate feedback and guidance of Buras to generate the guidance and instructions of Adi, as both references and the claimed invention are directed to augmented reality guidance systems for providing users guidance for using medical equipment. One of ordinary skill in the art would modify Adi by coding the system to include a machine learning model for analyzing the user and image data, generating the guidance and instructions, and generating the virtual avatar/images and video. Upon such modification, the method and system of Adi would include wherein the assistance scheme is computed by applying a machine learning tool; and/or wherein the output is generated by applying a machine learning tool. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Buras with Adi’s system and method in order to improve system performance and efficiency and to provide better and adaptive feedback and guidance to the user.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adi in view Cork et al. (US PGPub 20170039423), hereinafter referred to as Cork.
Regarding claim 18, Adi further teaches further comprising: receiving user input, wherein the user input comprises a gesture, a voice input, and/or a haptic input; wherein the assistance scheme is computed based on the received user input and/or the output is generated based on the received user input. However, Cork teaches a augmented reality device for use in a medical facility wherein the device can present instructions to the user including training instructions and wherein the augmented reality device can receive user voice commands or touch inputs (haptic) wherein the inputs can adjust settings, select a medical product for identification or troubleshooting, or advancing through training instructions (Paragraphs 0021, 0035, 0037, 0039-0040, 0042, 0048).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Adi to incorporate the teachings of Cork by incorporating a microphone and touch input into the augmented reality device and receiving user inputs to adjust settings and advance or select training instructions of Cork with the augmented reality goggles and generated guidance and instructions of Adi, as both references and the claimed invention are directed to augmented reality guidance systems for providing users guidance for using medical equipment. One of ordinary skill in the art would modify Adi by including a microphone and touch input to the augmented reality device and coding the system to receive user inputs and selections for settings, selecting a medical device/dialysis device to provide guidance on, and advancing or selecting the dialysis path/training instructions such that the generated guidance is based in part on the user input and/or selection. Upon such modification, the method and system of Adi would include further comprising: receiving user input, wherein the user input comprises a gesture, a voice input, and/or a haptic input; wherein the assistance scheme is computed based on the received user input and/or the output is generated based on the received user input. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Cork with Adi’s system and method in order to allow users to select their desired training and improve user interaction by allowing a user to advance through the instructions.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adi in view of Berriman et al. (US PGPub 20140078144), hereinafter referred to as Berriman, as supported by Mason et al. (US PGPub 20210134426), hereinafter referred to as Mason.
Regarding claim 20, Adi further teaches wherein the representation of the virtual character or person is adjustable by the user based on a selection out of a plurality of prestored templates. However, Berriman teaches systems and methods for modifying a virtual avatar including selecting a template avatar and modifying the avatar (Abstract; Paragraphs 0029, 0034), and Mason teaches a method and system for using virtual avatars during medical exercise sessions wherein the virtual avatar is a representation of a user and can be designed and modified by the user to look like their own self or any suitable person (Abstract; Paragraphs 0003, 0105, 0122).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Adi to incorporate the teachings of Berriman by including an avatar selection and design step of Berriman to generate the avatar of Adi, as, while the references are directed to different fields of endeavor, character/avatar creators and selection are well-known techniques to one of ordinary skill in the art and would be obvious to apply to Adi to improve the system in the same way by allowing a user to generate an avatar to their liking and/or to resemble themselves (Mason Paragraph 0122). One of ordinary skill in the art would modify Adi by coding the system to include an avatar selection and design upon a user’s first time using the system wherein the selection would include selectable avatar templates for the user to choose. Upon such modification, the method and system of Adi would include wherein the representation of the virtual character or person is adjustable by the user based on a selection out of a plurality of prestored templates. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Berriman with Adi’s system and method in order to allow users to customize the virtual avatar to look like themselves or a suitable person such as a medical professional (Mason Paragraph 0122).
Conclusion
Accordingly, claims 1-20 are rejected.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CORRELL T FRENCH whose telephone number is (571)272-8162. The examiner can normally be reached M-Th 7:30am-5pm; Alt Fri 7:30am-4pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kang Hu can be reached at (571)270-1344. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CORRELL T FRENCH/Examiner, Art Unit 3715
