DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed November 21, 2024 fails to comply with 37 CFR 1.98(a)(3)(i) because it does not include a concise explanation of the relevance, as it is presently understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about the content of the information, of each reference listed that is not in the English language. It has been placed in the application file, but the information referred to therein has not been considered.
The information disclosure statement filed November 21, 2024 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered.
Claim Objections
Claims 1-13 are objected to because of the following informalities:
Claims 1-13 are objected to for the capitalization of the word “CHARACTERIZED”.
Claims 1, 3, 5, 7-9, 11, and 12 are objected to for the use of the word “it”.
In claim 10, line 1, “said container box” should read “said box”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-11 and 13 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites the limitation “wherein said cameras and sensors are operatively connected to a software for generating a stereoscopic representation from the images taken by the camera” which, as a computer implemented invention/function, requires support for both the hardware and algorithm for performing the function. The specification fails to provide support for the algorithm for performing the function as the specification merely recites the limitation verbatim and does not recite or teach the steps/algorithm for generating a stereoscopic representation from the images taken. Therefore, one of ordinary skill in the art would not reasonably believe that the inventor or a joint inventor, at the time the application was filed, had possession of the claimed invention.
Claims 2-11 and 13 are rejected by virtue of their dependency from claim 1.
Claim 7 recites the limitation “uses a three-dimensional gaming and simulation environment created by means of simulation engines” which, as a computer implemented invention/function, requires support for both the hardware and algorithm for performing the function. The specification fails to provide support for the algorithm for performing the function as the specification merely recites the limitation verbatim and does not recite or teach the steps/algorithm for creating a three-dimensional gaming and simulation environment and the terminology of “uses a three-dimensional gaming and simulation environment” does not provide the use or function of the environment. Therefore, one of ordinary skill in the art would not reasonably believe that the inventor or a joint inventor, at the time the application was filed, had possession of the claimed invention.
Claims 8-9 and 13 are rejected by virtue of their dependency from claim 7.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the user’s senses" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the position and orientation" in line 11. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the user’s movements" in line 12. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the images" in line 16. There is insufficient antecedent basis for this limitation in the claim.
Claims 2-11 and 13 are rejected by virtue of their dependency from claim 1.
Claim 4 recites the limitation “internal electronic complements for stereoscopic representation of the cameras” which is not defined by the specification or claim in such a way that one of ordinary skill in the art would understand what the inventor or a joint inventor regards as the invention. The limitation “internal electronic complements” is not a known term of art or defined such that one of ordinary skill in the art would understand what the limitation entails. Therefore, the limitation renders the claim unclear and indefinite as one of ordinary skill would not be able to determine what the inventor or a joint inventor regards as the invention. For the sake of compact prosecution, the limitation is interpreted as electronic components of the headset.
Claim 5 recites the limitation “WEB architecture data storage system” which is not defined by the specification or claim in such a way that one of ordinary skill in the art would understand what the inventor or a joint inventor regards as the invention. The limitation “WEB architecture data storage system” is not a known term of art or defined such that one of ordinary skill in the art would understand what the limitation entails. Therefore, the limitation renders the claim unclear and indefinite as one of ordinary skill would not be able to determine what the inventor or a joint inventor regards as the invention. For the sake of compact prosecution, the limitation is interpreted as a network server storage/database.
Claim 6 recites the limitation "the … distance, and velocities" in line 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation "the user" in line 16. There is insufficient antecedent basis for this limitation in the claim.
Claim 9 recites the limitation “three-dimensional models” in line 2. The limitation is unclear as it is unclear if the models are computer/simulation models in the simulated environment or physical models in the box that correspond with a simulation object in the simulated environment and renders the claim indefinite as it is unclear what the inventor or a joint inventor regards as the invention.
Claim 11 recites the limitation "the user’s mobility" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 12 recites the limitation "the user’s sense" in line 5. There is insufficient antecedent basis for this limitation in the claim.
Claim 12 recites the limitation "the information" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Claim 12 recites the limitation "the simulations performed" in line 7. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
Claim(s) 12 is/are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Van Flute et al. (US PGPub 20210074183), hereinafter referred to as Flute.
With regard to claim 12, Flute teaches a device for simulation of minimal access surgeries by means of immersive virtual reality (Abstract; Paragraphs 0001, 0047 teach an augmented and mixed (virtual) reality simulator for minimally invasive surgical training), characterized in that it comprises:
a mechanical interface that connects to a screen for visualizing the simulation of a surgery process (Figs. 2, 4; Paragraphs 0049, 0051-0052 teach the system includes a physical simulator unit/box trainer that connects to a display unit (screen));
wherein said mechanical interface contains two handles that simulate surgical clamps for immersion of the user's senses (Paragraphs 0057-0058, 0076, 0081 teach the box includes two surgical tool entry openings that can receive two surgical tools (two handles) that simulate laparoscopic tools that the user manipulates); and
wherein said mechanical interface stores in an electronic device the information of the simulations performed (Paragraph 0109 teaches the system/simulator includes storing user progress on surgical training).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-10 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Van Flute et al. (US PGPub 20210074183), hereinafter referred to as Flute, in view of Fisher et al. (US PGPub 20180090029), hereinafter referred to as Fisher.
With regard to claim 1, Flute teaches an immersive virtual reality simulator-type system for minimal access surgery and other surgical procedures (Abstract; Paragraphs 0001, 0047 teach an augmented and mixed (virtual) reality simulator for minimally invasive surgical training), characterized in that it comprises:
a mechanical interface that connects to a screen for simulating a surgical procedure (Figs. 2, 4; Paragraphs 0049, 0051-0052 teach the system includes a physical simulator unit/box trainer that connects to a display unit (screen));
said mechanical interface contains two handles that simulate surgical clamps for immersion of the user's senses (Paragraphs 0057-0058, 0076, 0081 teach the box includes two surgical tool entry openings that can receive two surgical tools (two handles) that simulate laparoscopic tools that the user manipulates);
said mechanical interface contains an electronic system that operatively connects to a screen to generate a visual interface (Paragraphs 0049, 0055-0056, 0069-0072, 0096-0097 teach the system includes wiring for connecting the box trainer to the display unit (screen) including wiring within the embedded model wherein the system generates images and virtual representations (visual interface) on the display unit based on the user actions and manipulations of the model and tools);
wherein said visual interface contains graphics developed with motor exercises (Paragraphs 0049, 0059, 0112 teach the system can include different tasks (motor exercises) wherein each task is displayed via the screen/display unit wherein the generated imagery for a given task would be displayed on the display unit);
cameras that provide information on the position and orientation of the user's movements in real time (Paragraphs 0054, 0086, 0096-0099 teach the system includes a camera or two or more cameras capable of capturing the movements of the tools by the user in real-time);
wherein said cameras are operatively connected to a software for generating a stereoscopic representation from the images taken by the camera (Paragraphs 0086, 0097-0098 teaches the cameras are arranged to acquire stereoscopic information which is augmented by the system and displayed (stereoscopic representation)); and
wherein said visual interface has three-dimensional objects of geometries similar to biological and surgical objects (Paragraphs 0076, 0095-0097 teach the system includes physical tissue models (three-dimensional object) and the system generates an augmented video including representations of the tissue model and laparoscopic tools (surgical objects) which is displayed on the display unit).
Flute may not explicitly teach camera and sensors that provide information on the position and orientation of the user's movements in real time, wherein said cameras are placed in a virtual reality headset system, and the sensors are placed in the handles. However, Fisher teaches a system and method for simulating medical procedures using a simulation system including two tools and a physical simulator “box” including a VR headset worn by the user which can be an Oculus Rift, HTC Vive or Samsung Gear VR or AR headset such as a HoloLens and wherein the tools include motion and touch sensors for sensing motions of the user (Paragraphs 0016, 0034-0035, 0047, 0067-0068). It is noted that Fisher does not explicitly teach camera placed in the VR headset, but Page 3, paragraph 3 and Page 7, paragraph 1 of the instant application state the VR headset can be an Oculus Quest or HTC Vive which is taught by Fisher and therefore teaches the limitation implicitly in view of the specification of the instant application.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Flute to incorporate the teachings of Fisher by combining the VR headset and tool sensors of Fisher with the simulation system of Flute, as both references and the claimed invention are directed to laparoscopic simulation and training systems and methods including simulating a surgical procedure. One of ordinary skill in the art would modify Flute by including motion and touch sensors in the handles of the medical tools and including a VR headset worn by the user and including cameras for capturing movement data of the tools and user actions and coding the system to use the captured data to further generate and augment the simulation video. Upon such modification, the method and system of Flute would include camera and sensors that provide information on the position and orientation of the user's movements in real time, wherein said cameras are placed in a virtual reality headset system, and the sensors are placed in the handles. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Fisher with Flute’s system and method in order to further monitor user performance and actions and provide a more immersive simulation via a VR headset.
With regard to claim 2, Flute, as modified, further teaches characterized in that said mechanical interface is included in a portable case (Fig. 4; Paragraphs 0038, 0051-0053 teach the simulator is in the form of a box which can be portable).
With regard to claim 3, Flute, as modified, further teaches characterized in that it includes a system for importing models of different types of surgeries (Paragraphs 0047, 0064, 0067, 0093-0094 teach the system includes physical tissue models as the basis for the surgical experience/procedure wherein the model can be different types of anatomy that are incorporated in the augmented video such that the different types of models can be placed in the simulator to simulate different surgeries). For the sake of compact prosecution, interpreting the limitation of “importing models” more specifically as importing data from a network or database, Flute may not explicitly teach the limitation in this interpretation. However, Fisher further teaches the system can share selected medical procedures with the VR simulation system such tat the VR/AR simulation system renders an operating environment that corresponds to the selected procedure and wherein systems can share and connect across a network and transmit data including the simulation/procedure (Paragraphs 0039-0040, 0062, 0065).
As discussed above, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Flute to incorporate the teachings of Fisher by applying the technique of generating the virtual environment based on the selected procedure of Fisher with the simulation system of Flute, as both references and the claimed invention are directed to laparoscopic simulation and training systems and methods including simulating a surgical procedure. One of ordinary skill in the art would modify Flute by coding the system to have a user select the medical procedure and generate an operating environment accordingly including transmitting the simulations across a network. Upon such modification, the method and system of Flute would include characterized in that it includes a system for importing models of different types of surgeries. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Fisher with Flute’s system and method in order to provide more types of procedures and simulations and improve user interaction.
With regard to claim 4, Flute may not explicitly teach characterized in that the virtual reality headset contains internal electronic complements for stereoscopic representation of the cameras. However, as discussed above, Fisher teaches a system and method for simulating medical procedures using a simulation system including a VR headset worn by the user which can be an Oculus Rift, HTC Vive or Samsung Gear VR or AR headset such as a HoloLens (Paragraphs 0016, 0034-0035, 0047, 0067-0068). It is noted that “internal electronic complements” is not defined by the specification as discussed above and is interpreted as electronic components of the VR headset. Fisher does not explicitly teach the components of the VR headset, but Page 3, paragraph 3 and Page 7, paragraph 1 of the instant application state the VR headset can be an Oculus Quest or HTC Vive which is taught by Fisher and therefore Fisher teaches the limitation implicitly in view of the specification of the instant application.
As discussed above, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Flute to incorporate the teachings of Fisher by combining the VR headset and tool sensors of Fisher with the simulation system of Flute, as both references and the claimed invention are directed to laparoscopic simulation and training systems and methods including simulating a surgical procedure. One of ordinary skill in the art would modify Flute by including motion and touch sensors in the handles of the medical tools and including a VR headset worn by the user and including cameras for capturing movement data of the tools and user actions and coding the system to use the captured data to further generate and augment the simulation video. Upon such modification, the method and system of Flute would include characterized in that the virtual reality headset contains internal electronic complements for stereoscopic representation of the cameras. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Fisher with Flute’s system and method in order to further monitor user performance and actions and provide a more immersive simulation via a VR headset.
With regard to claim 5, Flute may not explicitly teach characterized in that it includes a WEB architecture data storage system. However, Fisher teaches the system includes a web server and database that the system can connect to and initiate to run the simulation (Paragraphs 0062, 0080-0084).
As discussed above, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Flute to incorporate the teachings of Fisher by including a web server and connecting the system to a network of Fisher with the simulation system of Flute, as both references and the claimed invention are directed to laparoscopic simulation and training systems and methods including simulating a surgical procedure. One of ordinary skill in the art would modify Flute by including network connections such that the system can connect to a web server or data server to handle processing functions including storing data from the database and processing the simulation remotely from the system. Upon such modification, the method and system of Flute would include characterized in that it includes a WEB architecture data storage system. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Fisher with Flute’s system and method in order to improve system performance and decentralize processing as server based processing is well-known in the art for handling computer program functions.
With regard to claim 6, Flute, as modified, further teaches characterized in that said software delivers information of physical parameters related to the position, orientation, distance, and velocities of the simulated clamps based on the maneuvers of the user on the handles (Paragraphs 0082, 0097 teach the camera is arranged to track and extract data of three dimensional movements of the tools which would include position, orientation, distance, and velocities of the tools (clamps) based on the user manipulation of the tools).
With regard to claim 7, Flute may not explicitly teach characterized in that it uses a three-dimensional gaming and simulation environment created by means of simulation engines. However, Fisher further teaches the system includes a VR simulation system including modules (simulation engines) that renders the operating room environment which is a virtual (three-dimension) simulation environment (Figs. 4b-d; Paragraphs 0040, 0048, 0050, 0052, 0072-0073).
As discussed above, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Flute to incorporate the teachings of Fisher by combining the VR headset and simulation system and environment of Fisher with the simulation system of Flute, as both references and the claimed invention are directed to laparoscopic simulation and training systems and methods including simulating a surgical procedure. One of ordinary skill in the art would modify Flute by including a VR headset worn by the user and coding the system to include a simulation system include modules for generating a virtual environment corresponding to the medical procedure. Upon such modification, the method and system of Flute would include characterized in that it uses a three-dimensional gaming and simulation environment created by means of simulation engines. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Fisher with Flute’s system and method in order to provide a more immersive simulation via a VR headset and further simulate the user’s environment.
With regard to claim 8, Flute may not explicitly teach characterized in that it includes three-dimensional models on said three-dimensional gaming and simulation environment. However, as discussed above, Fisher further teaches the system includes a VR simulation system including modules (simulation engines) that renders the operating room environment which is a virtual (three-dimension) simulation environment including surgical objects and a patient as seen in figs. 4b-f (Figs. 4b-f; Paragraphs 0040, 0048, 0050, 0052, 0072-0073).
As discussed above, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Flute to incorporate the teachings of Fisher by combining the VR headset and simulation system and environment of Fisher with the simulation system of Flute, as both references and the claimed invention are directed to laparoscopic simulation and training systems and methods including simulating a surgical procedure. One of ordinary skill in the art would modify Flute by including a VR headset worn by the user and coding the system to include a simulation system include modules for generating a virtual environment corresponding to the medical procedure including objects and models. Upon such modification, the method and system of Flute would include characterized in that it includes three-dimensional models on said three-dimensional gaming and simulation environment. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Fisher with Flute’s system and method in order to provide a more immersive simulation via a VR headset and further simulate the user’s environment.
With regard to claim 9, Flute, as modified, further teaches characterized in that it also includes a box containing three-dimensional models simulated in the three-dimensional gaming and simulation environment (Paragraphs 0051, 0067, 0084, 0087 teach the system includes a box trainer (box) that includes replaceable tissue models (three-dimensional models) which are captured as video data by the camera to generated the augmented/mixed reality video which would include the environment as discussed in the rejection of claim 9 above).
With regard to claim 10, Flute further teaches characterized in that said container box has dimensions in correspondence with the surgical procedure being simulated (Fig. 5; Paragraphs 0053, 0063, 0077 teach the physical simulator is in the form of a box having sides and removable panels (dimensions) and includes a removable physical tissue model wherein the model corresponds to the surgical procedure being simulated).
With regard to claim 13, Flute further teaches characterized in that said mechanical interface is included in a portable case (Fig. 4; Paragraphs 0038, 0051-0053 teach the simulator is in the form of a box which can be portable).
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Flute in view of Fisher as applied to claim 1 above, and further in view of Ribeira et al. (US PGPub 20170213473).
With regard to claim 11, Flute in view of Fisher may not explicitly teach characterized in that it includes a system for presenting results of the user's mobility in the simulations, wherein said results are presented visually on the screen or sent via WEB. However, Ribeira teaches a method and system for augmented reality simulation for educational training including medical/surgical procedures wherein the system records user actions and evaluates user performance based upon pre-determined metrics wherein the system generates an evaluation report (results of the user’s actions) that can be printed or transmitted over a network (sent via web) (Paragraphs 0026, 0033).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Flute in view of Fisher to incorporate the teachings of Ribeira by applying the technique of evaluating user performance and generating an evaluation report of Ribeira for the detected user performance/motions of Flute, as both references and the claimed invention are directed to simulation based training systems and methods including simulating a surgical procedure. One of ordinary skill in the art would modify Flute in view of Fisher by coding the system to evaluate the sensed user motion and actions and generate an evaluation report (result) for a selected/performed surgical procedure and present and/or transmit the report. Upon such modification, the method and system of Flute would include characterized in that it includes a system for presenting results of the user's mobility in the simulations, wherein said results are presented visually on the screen or sent via WEB. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate these teachings from Ribeira with Flute in view of Fisher’s system and method in order to evaluate user performance and improve user feedback to improve user performance.
Conclusion
Accordingly, claims 1-13 are rejected.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CORRELL T FRENCH whose telephone number is (571)272-8162. The examiner can normally be reached M-Th 7:30am-5pm; Alt Fri 7:30am-4pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kang Hu can be reached at (571)270-1344. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CORRELL T FRENCH/Examiner, Art Unit 3715
Prosecution Insights