DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the application filed on 11/27/24.
Claims1-17 are currently pending and have been examined.
Continuity/Priority Date
Status of this application as a 371 of PCT/US2023/024019, filed 05/31/2023, claiming priority to US Provisional Application 63/347,470, filed 05/31/2022 is acknowledge. Therefore, a priority date of 05/31/2022 has been given to the instant application.
Claim Objections
Claim 15 is objected to for minor informalities. Claim 15 recites, at lines 7-8 “that prioritizes placement a surgical instrument tray to which is assigned most- used actual surgical instruments”. The underlined portion appears to have accidentally omitted a word, likely between “placement” and “a”. For purposes of examination, it is being interpreted as “that prioritizes placement of a surgical instrument tray to which is assigned most- used actual surgical instruments”. Please correct or explain on the record.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 are rejected under 35 U.S.C.101 because the claimed invention is directed to a judicial exception (an abstract idea) without significantly more.
Step 1
Claims 1-15 are drawn to a method, Claim 16 is drawn to a computer-readable device, and Claim 17 is drawn to a system, each of which are within the four statutory categories. Claims 1-17 are further directed to an abstract idea on the grounds set out in detail below.
Step 2A Prong 1
Claim 1 recites implementing the steps of:
receiving user input that selects, from the different arrangements of surgical instrument trays, a selected arrangement of surgical instrument trays for use during a surgical procedure, the information identifying surgical instruments assigned to each surgical instrument tray from the selected arrangement of surgical instrument trays;
presenting a graphical representation of the selected arrangement of surgical instrument trays;
receiving image data depicting multiple actual surgical instrument trays that are located on one or more surfaces configured to hold surgical instrument trays, the image data depicting actual surgical instruments located on the multiple actual surgical instrument trays, the image data captured by one or more cameras positioned to view the one or more surfaces;
analyzing the image data to determine an extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays; and
presenting a graphical indication of the extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays.
These steps amount to managing personal behavior or relationships or interactions
between people and therefore recite certain methods of organizing human activity. The claims recite a series of personal behaviors or interactions between healthcare personnel to select an arrangement of surgical instrument trays, provide a graphical representation of the selected trays, receive image data of an actual tray, compare the actual tray image to the selected arrangement to determine the extent to which the trays match, and provide a visual indication of the extent to which the actual tray matches the selected tray.
Independent claims 16, 17 recites similar limitations and also recites an abstract idea under the same analysis.
The above claims are therefore directed to an abstract idea.
Step 2A Prong 2
This judicial exception is not integrated into a practical application because the additional
elements within the claims only amount to:
A. Instructions to Implement the Judicial Exception. MPEP 2106.05(f)
The independent claims additionally recite:
a display device as implementing the steps presenting a graphical representation of the selected arrangement of surgical instrument trays and presenting a graphical indication of the extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays (Claims 1, 16, 17)
a computing system as implementing the steps of receiving user input that selects from the different arrangements of surgical instrument trays, a selected arrangement of surgical instrument trays for use during a surgical procedure…; receiving image data depicting multiple actual surgical instrument trays that are located on one or more surfaces configured to hold surgical instrument trays…; and analyzing the image data to determine an extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays
one or more computer-readable devices including instructions that, when executed by one or more processors as causing the computer system to perform the operations of Claim 16
one or more processors as executing the instructions to cause the computer system to implement the steps of Claim 17
The broad recitation of general purpose computing elements (one or more processors, one or more computers, one or more storage devices storing instructions that are executed by the one or more computers, a patient database, a memory device, electronic medical health data, a display) at a high level of generality only amounts to mere instructions to implement the abstract idea using computing components as tools.
Regarding the display device, para. [0315] discloses the system being implemented on a computer with a display device (“e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor”) for displaying information to the user. No particulars of the display appear to be provided; therefore, this is understood to be a general purpose display on a computing device functioning in its ordinary capacity.
Regarding the computing system, para. [0298] discloses, “computing devices 9800, 9850 that may be used to implement the systems and methods described in this document, as either a client or as a server or plurality of servers. Computing device 9800 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device 9850 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other similar computing devices”; see also [0299] teaching on system architecture – processor, memory, storage device, etc. No particulars are provided; therefore the computing system is given its broadest reasonable interpretation of one or more general purpose computing devices functioning in their ordinary capacity.
Regarding the one or more computer-readable devices including instructions that, when executed by one or more processors, no particulars of this element appear to be disclosed. Per [0301], disclosing “floppy disk device, hard disk device, optical disk device, or a tape device, flash memory or other similar solid state memory device”), and [0314] (e.g., instructions for a programmable processor) this element is given its broadest reasonable interpretation as a general purpose computing element functioning in its ordinary capacity.
Regarding the one or more processors, per at least para. [0299], this is understood to be a general purpose computing processor functioning in its ordinary capacity to implement the steps pf the abstract idea.
B. Insignificant Extra-Solution Activity. MPEP 2106.05(g)
Claims 1, 16, 17 additionally recite
storing, by a computing system, information that identifies different arrangements of surgical instrument trays;
This step only amounts to insignificant extra-solution activity. As stated in MPEP 2106.05(g), "[t]he term "extra-solution activity" can be understood as activities incidental to the primary process or product that are merely a nominal or tangential addition to the claim." In the present claim, the function of s storing, by a computing system, information that identifies different arrangements of surgical instrument trays is only nominally or tangentially related to the process of selecting an arrangement of surgical instrument trays, comparing the selection to an image of actual surgical instrument trays, and determining and providing an indication of the extent to which the selected tray matches the actual tray, and accordingly constitutes insignificant extra-solution activity.
C. Generally Linking a Judicial Exception to a Field of Use/Technological Environment MPEP 2106.05(h)
Claim 17 additionally recites:
“a rack assembly that includes one or more surfaces adapted to receive surgical instrument trays” and
“an image sensor mounted to view the one or more surfaces”
These limitations only amount to generally linking the judicial exception to a particular technological environment or field of use. Examiner prospectively notes that in the instant claim, the recitation of “a rack assembly that includes one or more surfaces adapted to receive surgical instrument trays” does not impact how any other functions in the claim are performed; none of the limitations provide a nexus to limit or improve “a rack assembly” or provide structural limitations of a “rack assembly”. Therefore, recitation of the rack assembly only amounts to generally linking the judicial exception to a particular technological field/field of use. Regarding “an image sensor mounted to view the one or more surfaces”, Examiner prospectively notes that recitation of “an image sensor mounted to view the one or more surfaces” does not impact how any other functions in the claim are performed; none of the limitations provide a nexus to the image sensor, e.g., the image sensor is not recited as performing any steps. While the claim recites “receiving, by the computing system, image data depicting multiple surgical instrument trays…”, the claim does not actually recite that the image data was captured by the “image sensor mounted to view the one or more surfaces”. Therefore, recitation of an image sensor only amounts to generally linking the judicial exception to a particular technological field/field of use.
These elements are therefore not sufficient to integrate the abstract idea into a practical application. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually.
Claims 1, 16, 17 as a whole, are therefore directed to an abstract idea.
These elements are therefore not sufficient to integrate the abstract idea into a practical application. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually.
The above claims, as a whole, are therefore directed to an abstract idea.
Step 2B
The present claims do not include additional elements that are sufficient to amount to
more than the abstract idea because the additional elements or combination of elements amount to no more than a recitation of:
A. Instructions to Implement the Judicial Exception. MPEP 2106.05(f)
As explained above, claims 1, 16, 17 only recite the aforementioned computing elements as tools for performing the steps of the abstract idea, and mere instructions to perform the abstract idea using a computer is not sufficient to amount to significantly more than the abstract idea. MPEP 2106.05(f).
B. Insignificant Extra-Solution Activity. MPEP 2106.05(g)
Likewise, as explained above, the step of storing, by a computing system, information that identifies different arrangements of surgical instrument trays, only amounts to insignificant extra-solution activity.
C. Generally Linking a Judicial Exception to a Field of Use/Technological Environment MPEP 2106.05(h)
Likewise, as explained above, the incorporation of “a rack assembly..” and “an image sensor” in Claim 17 only amounts to generally linking the judicial exception to a particular technological environment or field of use. Limiting the use of the abstract idea to one particular field does not add significantly more.
D. Well-Understood, Routine and Conventional Activities. MPEP 2106.05(d)
In addition to amounting to insignificant extra-solution activity the elements in Section B above constitute well-understood, routine and conventional activity. The step of storing, by a computing system, information that identifies different arrangements of surgical instrument trays only amounts to storing/retrieving data in memory, which has been previously held to be well-understood, routine and conventional when claimed at a high level of generality or as insignificant extra-solution activity. See MPEP 2106.05(d)(II).
Thus, taken alone, the additional elements do not amount to significantly more than the
above-identified judicial exception. Looking at the limitations as an ordered combination adds
nothing that is not already present when looking at the elements taken individually. Their
collective functions merely provide conventional computer implementation.
Depending Claims
Dependent claims recite additional subject matter which further narrows or defines the abstract idea embodied in the claims.
Claim 2 recites limitations pertaining to wherein presenting the graphical representation of the selected arrangement of surgical instrument trays includes presenting multiple graphical elements that:(i) each represent a corresponding surgical instrument tray from the selected arrangement of surgical instrument trays; and(ii) are visually arranged within the graphical representation according to the selected arrangement of surgical instrument trays. This only further limits the scope of independent claim 1. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 3 recites limitations pertaining to wherein presenting the graphical representation of the selected arrangement of surgical instrument trays includes depicting, for each respective surgical instrument tray from the selected arrangement of surgical instrument trays, surgical instruments assigned to the respective surgical instrument tray. This only further limits the scope of independent claim 1. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 4 recites limitations pertaining to wherein the image data was captured by a camera that is mounted to: (i) a shelf assembly that provides the one or more surfaces; or (ii) a structure that is suspended over the shelf assembly. This only further limits the scope of independent claim 1. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 5 recites limitations pertaining to wherein: analyzing the image data includes determining that the multiple actual surgical instrument trays are arranged in a manner that does not match the selected arrangement of surgical instrument trays; and presenting the graphical indication includes presenting an indication that the multiple actual surgical instrument trays are arranged in the manner that does not match the selected arrangement of surgical instrument trays, which is also certain methods of organizing human activities, as performing an analysis and determination of whether or not two trays match, and subsequently providing a graphical indication that the trays do not match, are personal behaviors that could be performed by healthcare personnel. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 6 recites wherein presenting the indication that the multiple actual surgical instrument trays are arranged in the manner that does not match the selected arrangement of surgical instrument trays includes indicating, as part of the graphical representation of the selected arrangement of surgical instrument trays, a particular surgical instrument tray for which the image data does not depict a matching actual surgical instrument tray. This only further limits the scope of independent claim 1. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 7 recites limitations pertaining to wherein determining that the multiple actual surgical instrument trays are arranged in the manner that does not match the selected arrangement of surgical instrument trays includes: identifying, based on the image data, a tray identification insignia attached to a first actual surgical instrument tray of the plurality of actual surgical instrument trays; and determining that the tray identification insignia attached to the first actual surgical instrument tray does not match insignia assigned to a corresponding tray location within the selected arrangement of surgical instrument trays which is also certain methods of organizing human activities, as identifying an insignia and determining that the insignia does not match a corresponding insignia are personal behaviors that could be performed by healthcare personnel. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 8 recites limitations pertaining to wherein analyzing the image data to determine an extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays includes: determining, based on the image data, that a particular actual surgical instrument has been removed from a first actual surgical instrument tray of the plurality of actual surgical instrument trays; and wherein presenting the graphical indication includes presenting an indication that the particular actual surgical instrument has been removed from the first actual surgical instrument tray, which is also certain methods of organizing human activities, as determining an instrument has been removed from a first tray and providing an indicator to show the instrument has been removed are personal behaviors that could be performed by healthcare personnel. Claim 8 recites “the computer system” as implementing at least some of the steps and as discussed above with respect to Claim 1, only amounts to mere instructions to apply the abstract idea. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 9 recites limitations pertaining to wherein: the image data represents a sequence of images depicting the multiple actual surgical instrument trays and changes thereto over a period of time. This only narrows the scope of the abstract idea. Claim 9 also recites limitations pertaining to determining, based on the image data, that the particular actual surgical instrument has been returned to the first actual surgical instrument tray; and presenting an indication that the particular actual surgical instrument has been returned to the first actual surgical instrument tray which is also certain methods of organizing human activities, as determining an instrument has been returned from a first tray and providing an indicator to show the instrument has been returned are personal behaviors that could be performed by healthcare personnel. Claim 9 recites “the display device” as implementing at least some of the steps and as discussed above with respect to Claim 1, only amounts to mere instructions to apply the abstract idea. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 10 recites limitations pertaining to storing, by the computing system:(1) a first time that the first particular actual surgical instrument was determined to have been removed from the first actual surgical instrument tray and a second time that the first particular actual surgical instrument was determined to have been returned to the first actual surgical instrument tray; or (2) a duration of time between the first particular actual surgical instrument being determined to have been removed from the first actual surgical instrument tray and the first particular actual surgical instrument determined to have been returned to the first actual surgical instrument tray. This only amounts to insignificant extra-solution activity in the form of storing/retrieving data in memory, which has been recognized by the courts as well-understood, routine and conventional activity when claimed as insignificant extra-solution activity. See MPEP 2106.05(d). The above limitations are therefore not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 11 recites limitations pertaining to wherein analyzing the image data to determine the extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays includes: determining, based on the image data, that a particular actual surgical instrument is misplaced among the plurality of actual surgical instrument trays, due to the particular actual surgical instrument being missing from a first actual surgical instrument tray and instead located on a second actual surgical instrument tray, which is also certain methods of organizing human activities, as determining an instrument has been misplaced among trays is a personal behaviors that could be performed by healthcare personnel. Claim 11 recites “the computing system” as implementing at least some of the steps and as discussed above with respect to Claim 1, only amounts to mere instructions to apply the abstract idea. Claim 11 also recites wherein presenting the graphical indication includes presenting an indication that the actual surgical instrument is misplaced only further narrows the scope of the abstract idea. These limitations are not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 12 recites limitations pertaining to wherein presenting the indication that the actual surgical instrument is misplaced includes indicating, as part of the graphical indication presented, that the particular surgical instrument is located on the second actual surgical instrument tray instead of the first actual surgical instrument tray. This only further limits the scope of the abstract idea. Claim 12 recites “the display device” as implementing at least some of the steps and as discussed above with respect to Claim 1, only amounts to mere instructions to apply the abstract idea. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 13 recites limitations pertaining to wherein analyzing the image data to determine the extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays includes: providing the image data to a computational model that has been trained to identify surgical instruments located on surgical instrument trays, based on analysis of image data of surgical instruments located on surgical instrument trays. which is also certain methods of organizing human activities, as inputting data into a computational model to determine the extent to which trays match is a personal behavior that could be performed by healthcare personnel. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 14 recites limitations pertaining to determining an actual usage of surgical instruments during one or more surgical procedures, based on the analyzing of the image data; and presenting an indication that one or more surgical instruments represent least-used surgical instruments during the one or more surgical procedures, based on the analyzing of the image data, which is also certain methods of organizing human activities, as determining usage of surgical instruments based on data analysis and providing an indication representing least-used surgical instruments are personal behaviors that could be performed by healthcare personnel. Claim 14 recites “the display device or another user interface” as implementing at least some of the steps and as discussed above with respect to Claim 1, only amounts to mere instructions to apply the abstract idea. This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 15 recites limitations pertaining to determining an actual usage of surgical instruments during one or more surgical procedures, based on the analyzing of the image data; and presenting an indication of a different arrangement of surgical instrument trays that form the selected arrangement of surgical instrument trays, that prioritizes placement a surgical instrument tray to which is assigned most- used actual surgical instruments, which is also certain methods of organizing human activities, as determining usage of surgical instruments based on data analysis and providing an indication of an arrangement that prioritizes placement of most-used instruments are personal behaviors that could be performed by healthcare personnel. Claim 15 recites “the display device or another user interface” as implementing at least some of the steps and as discussed above with respect to Claim 1, only amounts to mere instructions to apply the abstract idea This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
The dependent claims have been given the full two-part analysis including analyzing the additional limitations both individually and in combination. The dependent claims, when analyzed individually, and in combination, are also held to be patent ineligible under 35 U.S.C. 101 as they include all of the limitations of claim 1. The additional recited limitations of the dependent claims fail to establish that the claims do not recite an abstract idea because the additional recited limitations of the dependent claims merely further narrow the abstract idea. Beyond the limitations which recite the abstract idea, the claims recite additional elements consistent with those identified above with respect to the independent claims which encompass adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claims 2-15 recite additional subject matter which amounts to additional elements consistent with those identified in the analysis of Claim 1 above. As discussed above with respect to Claim 1 and integration of the abstract idea into a practical application, recitation of these additional elements only amounts to invoking computers as a tool to perform the abstract idea. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation.
Dependent claims 2-15, when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitation(s) fail(s) to establish that the claim(s) is/are not directed to an abstract idea without significantly more. These claims fail to remedy the deficiencies of their parent claims above, and are therefore rejected for at least the same rationale as applied to their parent claims above, and incorporated herein.
For the reasons stated, Claims 1-17 fail the Subject Matter Eligibility Test and are consequently rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 8, 14, 16, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Canton et. al. (US Publication 20230386074A1) in view of Roh et. al. (US Patent 11583361B1).
Regarding Claim 1, Canton discloses
storing, by a computing system, information that identifies different arrangements of surgical instrument trays ([0075] teaches either a user input value corresponding to an expected list of surgical tool identities or using a code such as a 2D code (barcode) provided to the computer, where the code is related to a “stored list” of expected surgical tools for a procedure; the code is used to access a stored list of expected tools – interpreted as teaching that information identifying arrangements of tools on surgical trays is stored by a computing system; per [0080] the expected information may include number and identity of tools related to a “specific tray setup” defined by a surgeon’s preference card, type of procedure, specific OR and/or patient characteristics – a “specific tray setup” is interpreted as synonymous with “arrangements” of surgical instrument trays);
receiving, the information identifying surgical instruments assigned to each surgical instrument tray ([0075], teaching on the list of expected identifies of surgical tools being based on a user input value; [0080] teaches on the expected number and identity of surgical tools may be related to a “specific tray setup” (interpreted as arrangement) as defined by a surgeon’s preference card);
receiving, by the computing system, image data depicting actual surgical instrument tray[[s]] that are located on one or more surfaces configured to hold surgical instrument trays, the image data depicting actual surgical instruments located on the actual surgical instrument tray[[s]], the image data captured by one or more cameras positioned to view the one or more surfaces ([0063], teaching on the memory storing software for performing image analytics as described within the scope of Canton, e.g., a computing system is performing the analysis; [0077] teaches on using computer vision to identify and track tools at various steps, e.g., assembly; [0080] teaches on capturing one or more images of a plurality of surgical tools assembled on a tray; per Fig. 1d this is understood to include images depicting actual surgical instruments on the tray, e.g., scissors can be seen in the figure among other instruments; [0082] teaches on using computer vision to capture and analyze images of surgical tools in a perioperative environment);
analyzing, by the computing system, the image data to determine an extent to which the actual surgical instrument tray[[s]] and the actual surgical instruments located on the actual surgical instrument tray[[s]] match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays ([0058] teaches on performing image analysis; images may be processed and analyzed to compare location of tools; [0063], teaching on the memory storing software for performing image analytics as described within the scope of Canton, e.g., a computing system is performing the analysis; [0077] teaches on using computer vision to identify and track tools at various steps, e.g., assembly; [0078] teaches on using computer vision for image capture; cameras transmit captured images to a processor which compares the captured images to reference images to identify the instruments in the image; [0080] teaches on expected number and identity of surgical tools being related to a specific tray setup, such as defined by a surgeon’s preference card or type of procedure; see Fig. 1d which shows an indicator of where an expected instrument is missing, which is understood to teach that the image has been analyzed in order to make the determination of missing instrument; each missing tool indicated is interpreted as an extent to which the actual tray does not match the selected tray); and
presenting, by the display device, a graphical indication of the extent to which the actual surgical instrument tray[[s]] and the actual surgical instruments located on the actual surgical instrument tray[[s]] match the selected arrangement of surgical instrument tray[[s]] and the surgical instruments assigned to the selected arrangement of surgical instrument tray[[s]] (See Fig. 1 showing an indication of a correct location of where one or more surgical tools should be located; Examiner interprets the indicator showing a correct location of one or more tools as being a graphical indication of the extent to which an actual surgical tray matches the selected arrangement; each missing tool indicated is interpreted as an extent to which the actual tray does not match the selected tray).
Canton does not explicitly teach the following. Roh, which is directed to robotic surgical inventory management, teaches:
multiple surgical instrument trays (Col. 28, lines 40-43 teach on the system determining whether “each of the items in each of the multiple surgical supply trays selected for a surgical procedure is present”; Col 29 line 61 – Col 30 line 19 teach on pre-op module (interpreted as being part of a computer system);
receiving, by the computing system, user input that selects, from the different arrangements of surgical instrument trays, a selected arrangement of surgical instrument trays for use during a surgical procedure, the information identifying surgical instruments assigned to each surgical instrument tray from the selected arrangement of surgical instrument trays (Col. 29 line 61 – Col 30 line 19 teach on pre-op module (interpreted as being part of a computer system) receiving user selection of a supply tray, e.g., supply tray 404a, from a supply database that is to be assigned to a particular patient, procedure, date, time or location; for example, patient Jane Doe has a total knee replacement procedure and has generic supply tray ID#123 assigned to the procedure (per Col 29 lines 16-19, tray 404a is understood to be the generic supply tray ID #123); the pre-op module can receive an indication that another supply tray, e.g., 404b, should also be assigned to the present patient/procedure; where 404b is a supply tray for a total knee replacement tray procedure; pre-op module receives the selection of additional supply trays until there is an indication that no more additional supply trays are received for the patient/procedure; Examiner interprets selection of a “generic” surgery tray 404a plus total knee replacement tray 404b to read on “a selected arrangement of surgical instrument trays” where 404a (generic), 404b (total knee replacement specific) constitute the arrangement);
presenting, by a display device, a graphical representation of the selected arrangement of surgical instrument trays (Col 27 lines 50-55 teach on retrieving a supply list specifying items for surgical supply trays selected for a surgical procedure from the supply database; see Fig. 5; Examiner interprets a list of the various components of selected surgical trays to read on broadest reasonable interpretation of “graphical representation”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton with the teachings of Roh, to use the system of Canton with the multiple surgical instrument trays of Roh and receive a user selection of a particular arrangement of surgical trays where information identifies surgical instruments assigned to each surgical tray from the selected arrangement, because a single surgical procedure can have multiple supply trays assigned to it (Roh Col. 30 lines 6-7), and to present a graphical representation of the selected arrangement of trays to determine if any needed items are absent (Roh Col 27 lines 57-60).
Regarding Claim 8, Canton/Roh teach the limitations of Claim 1. Canton further discloses wherein analyzing the image data to determine an extent to which the actual surgical instrument trays and the actual surgical instruments located on the actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays includes ([0010], teaching on comparing the recorded identities of each surgical tool in a plurality of surgical tools to a list of expected identities of the plurality of surgical tools):
determining, by the computing system and based on the image data, that a particular actual surgical instrument has been removed from a first actual surgical instrument tray of the plurality of actual surgical instrument trays ([0011] teaches on the system determining that one or more surgical tools is missing based on the comparing step of [0010]); and
wherein presenting the graphical indication includes presenting an indication that the particular actual surgical instrument has been removed from the first actual surgical instrument tray ([0012] teaches on based on the comparing step, providing an alert when it is determine that one or more surgical tools are missing from the plurality of expected surgical tools; the alert may include information comprising an identity of each of the one or more missing tools; the alert may be provided as an overlay on the plurality of surgical tools).
Though Canton does not teach on “multiple” surgical trays, Roh teaches this element as shown above with respect to Claim 1: multiple surgical instrument trays (Col. 28, lines 40-43 teach on the system determining whether “each of the items in each of the multiple surgical supply trays selected for a surgical procedure is present”; Col 29 line 61 – Col 30 line 19 teach on pre-op module (interpreted as being part of a computer system) being used to select trays 404a and 404b for a specific procedure for a patient undergoing a total knee replacement).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh with the teachings of Roh, to use the system of Canton with the multiple surgical instrument trays of Roh and receive a user selection of a particular arrangement of surgical trays, because a single surgical procedure can have multiple supply trays assigned to it (Roh Col. 30 lines 6-7).
Regarding Claim 14, Canton/Roh teach the limitations of Claim 1. Canton further discloses comprising: determining an actual usage of surgical instruments during one or more surgical procedures, based on the analyzing of the image data ([0057] teaches on tracking usage of tools in an OR, e.g., which tools are used and which remain unused; [0111] further teaches on using captured image data of a plurality of surgical tools during a procedure, identifying the tools used and recording the identity of tools removed to form a list of surgical tools used during the aseptic procedure); and
presenting, by the display device or another user interface, an indication that one or more surgical instruments represent least-used surgical instruments during the one or more surgical procedures, based on the analyzing of the image data ([0057] teaches on tracking usage of tools in OR – which are used and which remain unused , and providing feedback regarding optimization of the surgical tools provided in a tray for each surgeon and/or procedure).
Regarding Claim 16, Canton/Roh teach the limitations of Claim 1. Claim 16 recites the same or substantially similar limitations as Claim 1, and the discussion above with respect to Claim 1 is equally applicable to Claim 16. Claim 16 additionally recites the following elements which are also disclosed by Canton: One or more computer-readable devices including instructions that, when executed by one or more processors, cause a computing system to perform operations ([0016], [0062] teaching on system architecture).
Regarding Claim 17, Canton/Roh teach the limitations of Claim 1. Claim 17 recites the same or substantially similar limitations as Claim 1, and the discussion above with respect to Claim 1 is equally applicable to Claim 17. Claim 17 additionally recites the following elements which are also disclosed by Canton:
a rack assembly that includes one or more surfaces adapted to receive surgical instrument trays ([0047] teaches on tools being placed onto racks, interpreted as a rack assembly with one or more surfaces to receive surgical instrument trays);
an image sensor mounted to view the one or more surfaces ([0051] teaches on using one or more cameras which may be mounted on a ceiling or poll extending from the ceiling, high up on a wall, for capturing images of perioperative environment which includes surgical instruments stored on trays per [0008]);
one or more processors ([0017], [0071], [0083]; and
one or more computer-readable devices including instructions that, when executed by the one or more processors, cause a computing system to perform operations ([0016], [0062] teaching on system architecture).
Claim(s) 2-5, 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Canton et. al. (US Publication 20230386074A1) in view of Roh et. al. (US Patent 11583361B1) as applied to Claim 1 above, and further in view of Gerstner (US Publication 20170202630A1).
Regarding Claim 2, Canton/Roh teach the limitations of Claim 1. Canton does not disclose, but Roh further teaches wherein presenting the graphical representation of the selected arrangement of surgical instrument trays includes presenting multiple graphical elements that: (i) each represent a corresponding surgical instrument tray from the selected arrangement of surgical instrument trays (Col. 29 line 61 – Col 30 line 19 teach on pre-op module for selection of multiple surgical instrument trays for a procedure, in this example, 404 a (generic) and 404b (total knee replacement); Col 27 lines 50-55 teach on retrieving a supply list specifying items for a surgical supply tray selected for a surgical procedure from the supply database; see Fig. 5 which shows list for Generic (left side) and list for Total Knee Tray (right side), interpreted as multiple graphical elements that each represent corresponding surgical instrument tray as the graphics show exactly which instruments and quantities are required for each selected tray).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh with these teachings of Roh to display multiple graphical elements that each represent a corresponding surgical instrument tray from the selected arrangement of surgical trays, with the motivation of determining whether any items are missing from at least one of the trays so that appropriate action can be taken to recover the missing item (Col 28 lines 22-30).
Canton/Roh do not teach, but Gerstner, which is directed to a surgical tray efficiency system, teaches:
wherein presenting the graphical representation of the selected arrangement of surgical instrument trays includes presenting multiple graphical elements that: (ii) are visually arranged within the graphical representation according to the selected arrangement of surgical instrument trays ([0170] teaches on a “planogram” for a given procedure that is customized to suit a particular surgeon’s needs; a planogram is an interactive digital display of representations of surgical instrument trays arranged on a vertical rack assembly. See Fig. 60, a tray view screen allowing a user to view data and media associated with a particular instrument tray; See Fig. 70 of verified planogram showing a selected arrangement of surgical trays including graphical elements that are visually arranged according to a selected arrangement of trays).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to further modify Canton/Roh/Gerstner with these teachings of Gerstner to present the graphical representation of the selected arrangement of trays by visually arranging the graphical elements according to the selected arrangement of trays, with the motivation of visually representing multiple trays in sequential order for use (Gerstner [0182]).
Regarding Claim 3, Canton/Roh/Gerstner teach the limitations of Claim 2. Canton does not disclose, but Gerstner further teaches: wherein presenting the graphical representation of the selected arrangement of surgical instrument trays includes depicting, for each respective surgical instrument tray from the selected arrangement of surgical instrument trays, surgical instruments assigned to the respective surgical instrument tray ([0057] teaches on a user inputting surgical planning data relating to a given procedure into a computer system to cause the display to present a particular aspect of the surgical planning data, where [0059] teaches that the surgical planning data may comprise surgical instrument tray content which indicates surgical instruments to be stored on various trays; where [0063] teaches on outputting an enlarged view of the particular surgical instrument tray as the particular aspect of the surgical planning data; see Fig. 62 showing enlarged image of an instrument tray where the position of individual instruments on the tray, e.g., scissors, can be observed; see [0170], teaching on the use of “planograms” to display a representation of surgical instrument trays; see Fig. 70 which includes a planogram screen showing the individual trays where each tray displays individual instruments on it).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to further modify Canton/Roh/Gerstner with these teachings of Gerstner, so that the graphical representation includes depicting instruments assigned to each respective surgical tray as taught by Gerstner, with the motivation of allowing the user to view images corresponding to instruments and trays (Gerstner [0173]).
Regarding Claim 4, Canton/Roh teach the limitations of Claim 1. Canton does not disclose, but Gerstner, which is directed to a surgical tray efficiency system, teaches wherein the image data was captured by a camera that is mounted to:(i) a shelf assembly that provides the one or more surfaces; or (ii) a structure that is suspended over the shelf assembly ([0205] teaches on an “identification assembly” which includes a camera and an attachment element to provide the support necessary to mount the identification assembly (e.g., including the camera) onto a vertical rack assembly containing surgical instruments).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh with these teachings of Gerstner to include a camera mounted to a structure above the shelf assembly with the motivation of taking pictures from above to populate a planogram (diagram for mapping tool locations per [0031]) (Gerstner [0205]).
Regarding Claim 5, Canton/Roh/Gerstner teach the limitations of Claim 4. Canton further discloses analyzing the image data includes determining that the actual surgical instrument trays are arranged in a manner that does not match the selected arrangement of surgical instrument trays ([0091] teaches on capturing images of surgical tools, recording the initial list of identities, and comparing the initial list of identifies of each surgical tool to an expected list of identities); and presenting the graphical indication includes presenting an indication that the actual surgical instrument trays are arranged in the manner that does not match the selected arrangement of surgical instrument trays ([0080] teaches on capturing image data of a plurality of surgical tools assembled on a tray, wherein the setup of the tray may be defined by a surgeon’s preference cards (selected arrangement); the device can be configured to output information regarding a missing surgical tool where an identity or correct location of a missing tool may be indicated to the display; see Fig. 1D).
Though Canton does not teach on “multiple” surgical trays, Roh teaches this element as shown above with respect to Claim 1: multiple surgical instrument trays (Col. 28, lines 40-43 teach on the system determining whether “each of the items in each of the multiple surgical supply trays selected for a surgical procedure is present”; Col 29 line 61 – Col 30 line 19 teach on pre-op module (interpreted as being part of a computer system) being used to select trays 404a and 404b for a specific procedure for a patient undergoing a total knee replacement).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh/Gerstner with the teachings of Roh, to use the system of Canton with the multiple surgical instrument trays of Roh and receive a user selection of a particular arrangement of surgical trays, because a single surgical procedure can have multiple supply trays assigned to it (Roh Col. 30 lines 6-7).
Regarding Claim 9, Canton/Roh teach the limitations of Claim 8. Canton further discloses wherein:
the image data represents a sequence of images depicting the multiple actual surgical instrument trays and changes thereto over a period of time ([0109] teaches on, during an aseptic procedure, repeating the capturing and processing of one or more images of the plurality of surgical tools substantially in real-time, the identity of each surgical tool removed from the plurality is recorded).
Canton/Roh do not teach, but Gerstner, which is directed to a surgical tray efficiency system, teaches:
the method comprises: determining, based on the image data, that the particular actual surgical instrument has been returned to the first actual surgical instrument tray ([0217]-[0218] teaches on returning a surgical instrument to its location on the tray after use; when the instrument is returned to its preferred location on the vertical rack assembly, the user interface may provide a graphical indication that the instrument is now stored on the vertical rack assembly – outputting such an indication infers that it has been determined that the surgical instrument has been returned); and
presenting, by the display device, an indication that the particular actual surgical instrument has been returned to the first actual surgical instrument tray ([0218] teaches on the user interface providing a graphical indication that an instrument returned to its preferred location on a tray and is now stored on the vertical rack assembly).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh with these teachings of Gerstner, to determine an instrument has been returned to the try and present an indication of it being returned via a display device, with the motivation of populating the planogram to show where instruments are stored and for instrument tracking purposes (Gerstner [0205]).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Canton et. al. (US Publication 20230386074A1) in view of Roh et. al. (US Patent 11583361B1) and further in view of Gerstner (US Publication 20170202630A1) as applied to Claim 5 above, and further in view of Tethrake et. al. (US Publication 20060119481A1).
Regarding Claim 6, Canton/Roh/Gerstner teach the limitations of Claim 5 but do not teach the following. Tethrake, which is directed to a workstation reader for surgical instruments and trays, teaches wherein presenting the indication that the multiple actual surgical instrument trays are arranged in the manner that does not match the selected arrangement of surgical instrument trays includes indicating, as part of the graphical representation of the selected arrangement of surgical instrument trays, a particular surgical instrument tray for which the [identification] data does not depict a matching actual surgical instrument tray ([0016] teaches on comparing actual information received for a surgical tray against stored information; [0017] teaches on analyzing surgical instrument trays as described in [0016]; the computer receives a signal indicating the result of the comparison of actual to expected tray contents; an LED on an LED panel of the workstation indicates the results: a green LED indicates favorable comparison and red LED indicates unfavorable comparison – red LED is interpreted as indicating the particular tray being analyzed does not match a selected/stored arrangement).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh with the teachings of Tethrake, to incorporate a means of indicating the image data of Canton/Roh for a particular tray does not match the selected arrangement of Canton/Roh by using a graphical representation to indicate the particular tray, with the motivation of indicating a tray that does not pass so a course of action can be taken to return the tray to passing status (Tethrake [0049]).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Canton et. al. (US Publication 20230386074A1) in view of Roh et. al. (US Patent 11583361B1) and further in view of Gerstner (US Publication 20170202630A1) as applied to Claim 5 above, and further in view of Schram et. al. (US Patent 20150166210 ).
Regarding Claim 7, Canton/Roh/Gerstner teach the limitations of claim 5. Canton does not disclose, but Gerstner further teaches wherein determining that the multiple actual surgical instrument trays are arranged in the manner that does not match the selected arrangement of surgical instrument trays includes: identifying, based on the image data, a tray identification insignia attached to a first actual surgical instrument tray of the plurality of actual surgical instrument trays ([0150] teaches on location indicia attached to the left side of a portion of a tray pad; [0151] teaches on location indicia placed on a tray which may include letters, numbers, symbols, shapes, colors and/or words alone or in combination – interpreted as a tray identification insignia).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh/Gerstner with these teachings of Gerstner, to identify a tray identification insignia attached to a surgical instrument tray, with the motivation of using the location indicia (“insignia”) to quickly and easily convey information to a user, e.g., a color could indicate a shelf location (Gerstner [0151]).
Canton/Roh/Gerstner do not teach the following, but Krishnamurthy, which is directed to item tracking for racks using digital image processing, teaches:
determining that the identification insignia attached to the first [item] does not match insignia assigned to a corresponding location (Fig 3 showing an identification insignia of a leaf being used to match a product to a location; Fig. 4 showing different storage locations of different products with different identification insignia (leaf, pumpkin; Examiner submits that if one can determine that the leaf matches Product A in Fig 4 while the pumpkin matches product C, it would be obvious that they could also determine that leaf does not match pumpkin and vice versa).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Canton/Roh/Gerstner with the teachings of Schram, to determine whether or not the identification insignia of Canton/Roh/Gerstner attached to the surgical tray of Gerstner matches an insignia assigned to a corresponding location as taught by Schram, within the selected arrangement of surgical trays (Roh), with the motivation of using a simple visual icon to reduce matching complexity (Schram [0038]).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Canton et. al. (US Publication 20230386074A1) in view of Roh et. al. (US Patent 11583361B1), and further in view of Gerstner (US Publication 20170202630A1) as applied to Claim 9 above, and further in view of Deland (US Publication 20130113929A1).
Regarding Claim 10, Canton/Roh/Gerstner teach the limitations of claim 9. Deland, which is directed to systems and methods for surgical procedure safety, teaches: comprising: storing, by the computing system:(1) a first time that the first particular actual surgical instrument was determined to have been removed from the first actual surgical instrument tray and a second time that the first particular actual surgical instrument was determined to have been returned to the first actual surgical instrument tray ([0057] teaches on tracking surgical instruments during a procedure; the surgical instrument tracker records the time at which a surgical instrument is removed from a tray; the time removed is recorded (“stored”); when a surgical instrument is placed back on the tray, the surgical instrument tracker identifies and records the time the instrument was placed back (returned) to the tray); or
(2) a duration of time between the first particular actual surgical instrument being determined to have been removed from the first actual surgical instrument tray and the first particular actual surgical instrument determined to have been returned to the first actual surgical instrument tray (per claim construction “or”, this limitation is not required, however, Examiner submits that one of skill in the art would recognize that if the time an instrument is removed from the tray and the time the instrument is returned to the tray, as taught by Deland, it would be possible to calculate the elapsed time, e.g., duration of time between instrument removal and return).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Canton/Roh with these teachings of Deland, to store the times at which a surgical instrument was removed from a tray and placed back onto the tray, with the motivation of ensuring instruments have been returned to the tray and are not unintentionally left in the body of a patient during the course of surgery to prevent consequences of retained surgical instruments (Deland [0002]).
Claim(s) 11-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Canton et. al. US Publication 20230386074A1 in view of Roh et. al. (US Patent 11583361B1) and further in view of Gerstner (US Publication 20170202630A1) as applied to Claim 9 above, and further in view of Krishnamurthy (US Publication 20230076979).
Regarding Claim 11, Canton/Roh/Gerstner teach the limitations of claim 9. Canton further discloses wherein analyzing the image data to determine the extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays includes: determining, by the computing system and based on the image data, that a particular actual surgical instrument is misplaced among the plurality of actual surgical instrument trays ([0010]-[0012] teach on using received image data of surgical tools, comparing the identities of identified tools to a list of expected tools and determining that a particular instrument is missing) due to the particular actual surgical instrument being missing from a first actual surgical instrument tray([0010]-[0012]; para. [0080], teaching on outputting via the display an indication of an identity and/or correct location of one or more missing surgical tools – the empty spot and indicator of missing tool is interpreted as the particular instrument is missing from the first tray); and wherein presenting the graphical indication includes presenting an indication that the actual surgical instrument is misplaced (para. [0080], teaching on outputting via the display an indication of an identity and/or correct location of one or more missing surgical tools; Fig. 1d indicates location and identity of a missing tool; a missing tool is interpreted as synonymous with misplaced instrument).
Canton does not disclose, but Krishnamurthy, which is directed to item tracking for racks using digital image processing, teaches:
determining, by the computing system and based on the image data, that a particular item is misplaced among the plurality of locations due to the particular item being missing from a first location and instead located on a second location ([0021] teaches on a master template identifying rack positions corresponding to a shelf of the rack and a location on the shelf where an item is placed; the image processing engine determines whether locations of items in the images of the rack match the locations in master template; a mismatch between the locations of items in the images and locations of items in the master template indicates that one or more items are in the wrong location on the rack; [0024] teaches on generating an image of a rack, identifying items located in the image of the rack, determining locations of items with respect to the rack in the image, comparing the locations of items in the image to locations defined by a master template; Per Fig. 3A, the different shelves storing inventory on the rack are interpreted as being synonymous with trays on a rack).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Canton/Roh/Gerstner with these teachings of Krishnamurthy to determine that a missing instrument from the first tray of Canton/Roh is located in a second location (e.g., a second tray of Roh) with the motivation of providing a notification to a user that one or more items (e.g., surgical instruments of Canton/Roh) are in the wrong location (Krishnamurthy [0049]).
Regarding Claim 12, Canton/Roh/Gerstner/Krishnamurthy teach the limitations of Claim 11. Canton does not disclose, but Krishnamurthy further teaches wherein presenting the indication that the actual [item] is misplaced includes indicating, as part of the graphical indication presented by the display device, that the particular [item] is located on the second actual [location] ([0021] teaches on a master template identifying rack positions corresponding to a shelf of the rack and a location on the shelf where an item is placed; the image processing engine determines whether locations of items in the images of the rack match the locations in master template; a mismatch between the locations of items in the images and locations of items in the master template indicates that one or more items are in the wrong location on the rack; [0024] teaches on generating an image of a rack, identifying items located in the image of the rack, determining locations of items with respect to the rack in the image, comparing the locations of items in the image to locations defined by a master template; Per Fig. 3A, the different shelves storing inventory on the rack are interpreted as being synonymous with trays on a rack).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Canton/Roh/Gerstner/Krishnamurthy with these teachings of Krishnamurthy to determine that a missing instrument from the first surgical tray of Canton/Roh is located in a second location (e.g., a second tray of Roh) with the motivation of providing a notification to a user that one or more items (e.g., surgical instruments of Canton/Roh) are in the wrong location (Krishnamurthy [0049]).
Regarding Claim 13, Canton/Roh/Gerstner/Krishnamurthy teach the limitations of claim 12. Canton further discloses wherein analyzing the image data to determine the extent to which the multiple actual surgical instrument trays and the actual surgical instruments located on the multiple actual surgical instrument trays match the selected arrangement of surgical instrument trays and the surgical instruments assigned to the selected arrangement of surgical instrument trays includes: providing the image data to a computational model that has been trained to identify surgical instruments located on surgical instrument trays, based on analysis of image data of surgical instruments located on surgical instrument trays (Abstract teaches on using a machine learning algorithm (interpreted as reading on “computational model”) to process images and determine identities of surgical tools; [0015] teaches on using a machine learning algorithm to evaluate captured images and output information pertaining to an identity and/or correct location of one or more missing tools; [0055] teaches on the computer vision being linked to a platform utilizing machine learning to optimize the process; the computer program is configured to construct a model based on example inputs, e.g., through supervised learning; [0057] teaches on using the learning engine to automatically and continuously identify and track specific tools; [0076] teaches on using a machine learning algorithm to train the system to recognize individual surgical tools).
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Canton et. al. (US Publication 20230386074A1 in view of Roh et. al. (US Patent 11583361B1) as applied to Claim 14 above, and further in view of Further in view of Jaynes et. al. (US Publication 20210292091A1).
Regarding Claim 15, Canton/Roh teach the limitations of Claim 14. Canton further discloses, comprising: determining an actual usage of surgical instruments during one or more surgical procedures, based on the analyzing of the image data ([0057] teaches on tracking usage of tools in an OR, e.g., which tools are used and which remain unused; [0111] further teaches on using captured image data of a plurality of surgical tools during a procedure, identifying the tools used and recording the identity of tools removed to form a list of surgical tools used during the aseptic procedure).
Canton does not disclose, but Jaynes, which is directed to configuration of customizable trays for storing articles used in the medical field, teaches presenting, by the display device or another user interface, an indication of a different arrangement of trays that form the selected arrangement of strays, that prioritizes placement of a tray to which is assigned most- used actual [articles] ([0058] teaches on optimizing layout of contents of several trays across several trays; [0083] teaches on identifying tray location to improve efficiency; a tray may be moved based on a frequency of access so that more frequently accessed trays may be moved to positions that are more easily and efficiently accessed to minimize movement when retrieving articles; per [0031] trays are understood to store “articles”; per [0032] the recommended arrangement may be output to a display device).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Canton/Roh with these teachings of Jayne, indicate a different arrangement of the surgical trays of Canton/Roh according to frequency of access, with the motivation of optimizing the layout for speed of access and retrieval of articles (Jayne [0079]), e.g., the instruments of Canton/Roh.
Conclusion
In the interest of expediting prosecution, Examiner respectfully requests that Applicant provides citations to relevant paragraphs of specification for support for amendments in future correspondence.
The following relevant prior art not cited is made of record:
US Publication 20210019532A1, teaching on a method and system for facilitating tray management before/after tool management
US Publication 20220292815A1, teaching on systems and methods for dynamic identification of a surgical tray and the items contained thereon
US Publication 20190328460A1, teaching on a navigation system for instrument tracking during a surgical procedure
US Patent 6158437 teaching on a surgical instrument support tray which includes indicia to indicate the general sequence in which instruments will be employed in a particular procedure
US Publication 20180204323A1, teaching on detection of surgical instruments on surgical tray to determine if instruments are missing
US Publication 20150250551A1, teaching on surgical asset tracking trays
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANNE-MARIE K ALDERSON whose telephone number is (571)272-3370. The examiner can normally be reached on Mon-Fri 9:00am-5:00pm EST and generally schedules interviews in the timeframe of 2:00-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fonya Long, can be reached on 571-270-5096. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNE-MARIE K ALDERSON/Primary Examiner, Art Unit 3682
Prosecution Insights