DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 19 August 2026 has been entered.
Formal Matters
Applicant’s response and amendments filed 14 August 2026 are acknowledged. Claims 5, 11, and 20 are cancelled. Claims 1, 4 and 8 are currently amended. Claims 1-4, 6-10, 12-19, and 21 are pending and under examination. Claims 1 and 4 are independent claims.
Objections/Rejections Withdrawn
The objection to claim 8 is withdrawn in light of Applicant’s amendments.
The rejection of claims 1-4, 6-10, 12-19 and 21 under 35 U.S.C. 103 as being unpatentable over Uyama et al., WO 2019181632 (26 September 2019) in view of Makrinich et al., US 20210313050 (8 October 2021), is withdrawn in light of Applicant’s amendments. However, Applicant’s amendments require modified rejection in light of the prior art as set forth below.
Response to Arguments
Applicant argues that the amended claims overcome the objections, rejections of record under 35 USC 112 and 101, (Remarks pp. 1-2). Applicant’s amendments are persuasive. However, modified rejections are set forth below.
Regarding Applicant’s arguments as to the functional steps of the claims, Applicant is reminded that the instant claims are drawn to a system with an intended use in the preamble of “for capturing records of a surgical procedure”. The system of independent claim 1 requires a camera and at least one processor. The system of independent claim 4 requires a camera, at least one processor, and a display. The remaining limitations are drawn to the functionality of the processor. The modified rejection sets forth the teachings, suggestions, and motivations of the prior art that are most applicable to the claims and the functional recitations, as written. All of the claims recite computer-implemented functionality, which has been addressed with specificity as to the particular limitations in the claims.
It is recognized that the amendments to the independent claims provide additional computer-implemented context as to the workflow processes. However, one of ordinary skill in the art also recognizes that computer-implemented processes are results-effective variables which can be optimized when using a general computer. The system encompasses both hardware and software. One of ordinary skill in the art would have had a reasonable expectation of success to modify software with workflow processes based on the modified rejections provided below in response to Applicant’s arguments and amendments. Such processes can be optimized by a person of ordinary skill in the art without undue experimentation based on the teachings of the references as set forth below. The references themselves provide the teaching, suggestion, and motivation for computer-implemented workflow processes in a healthcare setting where modifications thereof would amount to nothing more than routine experimentation that can be optimized on an individual use case basis. See, In re Antonie, 559 F.2d 618, 195 USPQ 6 (CCPA 1977) and In re Boesch, 617 F.2d 272, 205 USPQ 215 (CCPA 1980)).
Modified Rejections – Necessitated by Amendment
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6-10, 12-19 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Uyama et al., WO 2019181632 (26 September 2019) in view of Makrinich et al., US 20210313050 (8 October 2021), and further in view of Squilla et al., US 20090182577 (16 July 2009).
Regarding currently amended independent claim 1, Uyama teaches a system (FIG 28; ¶9, apparatus) for capturing records of a surgical procedure (¶22), comprising: a camera (FIG 1, imager 11; ¶18) positionable to capture real-time image data (claim 12; ¶19, real-time) of a surgical treatment site (claim 8; ¶¶19-20); at least one processor (claim 11; FIG 3, information processing device 130); and at least one memory storing instructions executable by said at least one processor (FIG 28; claim 21; ¶198) to:
receive the real-time image data (¶81 “Live”) captured within the surgical instrument site (claim 21) and, when available, kinematic information from a robotic component maneuvering a surgical instrument (claim 21; ¶¶51-54);
analyze the real-time image data (¶55, SLAM mapping) to detect and identify at least one surgical instrument (¶¶29-30) at the surgical treatment site and to track motions of the identified surgical instrument (¶29);
Uyama does not expressly teach determining, based on whether the tracked motions include tool movement patterns or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure, that a predetermined surgical step has been performed or completed and in response to determining that the predetermined surgical step has been completed, automatically extract from the real-time image data a still image captured at a time of completion of said predetermined surgical step and store the still image in the at least one memory in a medical record in association with the predetermined surgical step.
Uyama teaches capturing and storing a still image (FIG 14; ¶31) annotating images while tracking specified positions on the basis of the position/posture information (¶100).
Makrinich teaches algorithmic determinations (¶203), based on whether the tracked motions include tool movement patterns (¶203) or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure (¶212), that a predetermined surgical step has been performed (¶¶171, 205) or completed (¶212) and in response to determining that the predetermined surgical step has been completed (¶¶212, 219), automatically extract from the real-time image data a still image captured at a time of completion of said predetermined surgical step and store the still image (surgical video frame, ¶213) in the at least one memory in a medical record in association with the predetermined surgical step (¶¶212, 213).
Makrinich also teaches systems comprising at least one processor (¶50) suitable for executing instructions pre-loaded into a memory integrated with or embedded into the controller or stored in a separate memory (¶50). One or more cameras 115 configured to track and identify a surgical tool within an anatomical structure via a computer-based camera control application that uses image recognition algorithms for positioning the camera to capture video/image data of a region-of-interest (ROI) or by a human operator controlling the position of the cameras are taught at ¶61. Surgical video may be recorded in real-time (¶185).
Neither Uyama nor Makrinich teach wherein said plurality of stored tasks or subtasks includes tasks or subtasks designated as ones for while photo documentation is to be captured upon completion thereof, and wherein said predetermined surgical step is a task or subtask so designated; and where the still image is saved independently of any video of the procedure that is stored.
Squilla teaches an automated information management healthcare facility workflow process with surgery-specific preferences 30 (FIGs 1, 3). The healthcare workflow process (FIG 1) is taught as encompassing patient photos and other information “content” related to a medical or surgical procedure (¶17). Content collected within the operating room is taught as including live video, radiological images, and photographs (¶19). FIG 2 illustrates modular information management system 10 that includes the collection, organization, and display of content (¶24). Squilla teaches that system 10 includes controller 12 connected to one or more storage devices 14, one or more content collection devices 16, one or more operator interfaces 18, one or more display devices 24, one or more remote receivers/senders 22 via one or more connection lines 28 (¶25). Content collection devices 16 are configured to retrievably store and/or archive captured content from videos and still images (¶29) and the stored content can be transferred from the content collection devices 16 to the storage device 14 and/or controller 12 during the collection phase of FIG 1 (¶29). Content collection devices 16 can locally store still images and/or videos and can be configured to later upload the substantially live content to the storage device 14 and/or the controller 12 (¶31).
Squilla teaches that the desired content is established (predetermined) at step 30 (FIG 3). Checklists that include all of the heterogeneous content needed (predetermined) are taught as being able to be verified within the “Collection Phase” (FIG 3; ¶41). If no new content is required, the checklist can be determined to be complete and the collected content saved (¶41). The workflow then proceeds to the “Organize Phase” (FIG 4, ¶49). Once those steps have been completed for each selected input in a key phase of a surgical sequence, the associated functionality of that sequence can be saved and the user may proceed to the next phase, the Display Phase (FIG 5; ¶¶50-51).
Squilla teaches universal functionality icons 96 comprising tools configured to access images captured during a surgical procedure (FIGs 6-8, ¶54). Rule set 118 is taught as “governing at least one of the collection phase (FIG. 3), organize phase (FIG. 4), and display phase (FIG. 5) of an exemplary healthcare facility workflow process” and can be based on at least one of many decision factors 120 (FIG 9, ¶69). Decision factors 120 are include doctor-specific preferences 124 comprising the organization of the surgical sequence phases, priority levels, display locations, content desired or requested and determinations regarding content collected and relevancy (FIG 9, ¶72). Content characteristics 122 can include specialist-indicated relevancy determinations which can be utilized as a factor in creating rule set 118 (¶69). Rule set 118 can be utilized to assist in automatically processing a plurality of content (¶77). Hierarchies can be applied to content types including live video and still images (¶84). An automated healthcare facility workflow process can be utilized to suggest to the user a preferred/optimized display layout displaying the plurality of content associated with a surgical procedure (¶84). FIG 9 shows steps by which the content can be organized by automatically associating content-specific functionality with each content of the plurality of content, which can be based on decision factors 120 (¶87).
Accordingly, Squilla teaches the claimed plurality of stored tasks or subtasks (FIGs 1, 3, 6-8, 9, workflow processes) includes tasks or subtasks (FIG 9, hierarchical organized content characteristics 122, surgical sequence (as doctor-specific preferences) 124, or specialty/surgical sequence preferences 126, or institution characteristics 128, or medical payer requirements for documentation 129) designated (FIG 9, by rule set 118) as ones for which photo documentation (content collection 102/132; still images ¶¶70, 84) is to be captured upon completion thereof (FIG 9, organized phase 134), and wherein said predetermined surgical step is a task or subtask so designated (FIG 9, rule set 118; claims 1, 9, 12), where the still image is saved independently of any video of the procedure that is stored (“still images”, ¶¶70, 84).
It would have been obvious to one having ordinary skill in the art as of the effective filing date of the invention to combine the teachings of Uyama, Makrinich, and Squilla given that the prior art included each element claimed, although not necessarily in a single reference. Uyama, Makrinich, and Squilla teach in the same field of endeavor, systems comprising processors, memory, and imaging systems for capturing information and making records of surgical procedures.
Although, Uyama discloses the claimed base processor, memory, camera, real-time data capture, and real-time analysis, including capturing and storing a still images, and annotating images while tracking specified positions on the basis of the position/posture information, Uyama does not disclose the functions of determining, based on whether the tracked motions include tool movement patterns or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure, that a predetermined surgical step has been performed or completed and in response to determining that the predetermined surgical step has been completed, automatically extract from the real-time image data a still image captured at a time of completion of said predetermined surgical step and store the still image in the at least one memory in a medical record in association with the predetermined surgical step.
Makrinich specifically addresses algorithmic determinations (¶203), based on whether the tracked motions include tool movement patterns (¶203) or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure (¶212), that a predetermined surgical step has been performed (¶205) or completed (¶212) and in response to determining that the predetermined surgical step has been completed (¶212), automatically extract from the real-time image data a still image captured at a time of completion of said predetermined surgical step and store the still image (surgical video frame, ¶213) in the at least one memory in a medical record in association with the predetermined surgical step (¶¶212, 213).
Squilla specifically addresses the claimed plurality of stored tasks or subtasks (FIGs 1, 3, 6-8, 9, workflow processes) includes tasks or subtasks (FIG 9) designated (FIG 9, rule set 118) as ones for which photo documentation (content collection 102/132; still images ¶¶70, 84) is to be captured upon completion thereof (FIG 9, organized phase 134), and wherein said predetermined surgical step is a task or subtask so designated (FIG 9, rule set 118; claims 1, 9, 12), where the still image is saved independently of any video of the procedure that is stored (“still images”, ¶¶70, 84).
Uyama includes the apparatus components of the system required to perform the functions of tool tracking movement patterns, steps or sequences of movements, capturing and storing a still image, annotating images while tracking specified positions on the basis of the position/posture information, and utilizing SLAM mapping, a person of ordinary skill in the art, seeking to utilize Uyama’s hardware architecture and SLAM mapping features for medical record collection, data analytics, and determining that a predetermined step in a surgical plan has been completed based on the imaging data captured during the procedure would reasonably consult Makrinich’s medical record and surgical workflow documentation solution. Makrinich’s use of algorithmic determinations and analytics can be incorporated alongside Uyama’s hardware (same one or more processors, instructions stored in at least one memory executable by the at least one processor, and camera(s) positionable to capture real-time image data of a surgical treatment site), using known methods without redesigning Uyama’s core architecture. Additionally, a person of ordinary skill in the art seeking to utilize Uyama’s hardware architecture and Makrinich’s analytics, could readily incorporate Squilla’s automated healthcare facility workflow processes, including a rules set (FIG 9, 118) that can be assigned and determined by any or all of the decision factors 120 (FIG 9) which can then automatically process content (collect, organize, and display/save/archive/collaborate) in order to automate the image capturing process of multiple individuals and groups in one workflow system. One of ordinary skill in the art would be motivated to add Squilla’s automated solution because it is modular, encompasses automatic processing of a plurality of content from a heterogenous content sources automating the workflow based on a rule set comprising hierarchical organization which encompasses still images that are displayed, saved, and archived.
Because the references address the same engineering problem (documenting real-time surgical imaging using at least one processor, at least one memory, and a camera positionable to capture real-time image data of a surgical treatment site) and the proposed modifications are mechanically compatible and implemented by routine engineering practices (adding software functions based on tracked motions including tool movement patterns or sequential surgical procedure steps, documenting that the predetermined surgical step has been completed and automatically extracting a still image (frame) from the real-time image data captured at a time of completion of a surgical step and storing the still images (frames) in the at least one memory in a medical record in association with the predetermined surgical step, where the steps are part of a healthcare facility workflow process that is automated and designed to utilize a rules-based platform for plural content from heterogeneous content sources), a person of ordinary skill in the art before the effective filing date of the claimed invention would have had a reasonable expectation of success in combining these teachings because they all work together as demonstrated by Squilla.
One of ordinary skill in the art recognizes that computer-implemented processes are results-effective variables which can be optimized when using a general computer. The system encompasses both hardware and software. One of ordinary skill in the art would have had a reasonable expectation of success to modify software with workflow processes based on the modified rejections provided below in response to Applicant’s arguments and amendments. Such processes can be optimized by a person of ordinary skill in the art without undue experimentation based on the teachings of the references as set forth below. The references themselves provide the teaching, suggestion, and motivation for computer-implemented workflow processes in a healthcare setting where modifications thereof would amount to nothing more than routine experimentation that can be optimized on an individual use case basis. See, In re Antonie, 559 F.2d 618, 195 USPQ 6 (CCPA 1977) and In re Boesch, 617 F.2d 272, 205 USPQ 215 (CCPA 1980)).
Regarding claim 2, Uyama modified by Makrinich and Squilla teaches the system of claim 1, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the instructions are further executable by said at least one processor (¶203) to, upon detecting completion of the predetermined surgical step (¶¶171, 205), prompting a user to dictate a note, recording a dictated note (audible prerequisite, ¶227), and storing said dictated note in the at least one memory in association with the predetermined surgical step (¶224).
Regarding claim 3, Uyama modified by Makrinich and Squilla teaches the system of claim 1, as set forth above, for the reasons set forth above.
Uyama teaches wherein the system further includes an image display (display device 15, ¶20) displaying real time (¶19) images from the camera (¶19), and wherein said instructions are further executable by said at least one processor (claim 22) to display the still image as an overlay on the display of the real time image (FIG 7, S14; FIG 8, S19).
Regarding currently amended independent claim 4, Uyama teaches a system (¶9, apparatus) for assisting a surgical practitioner (¶3), comprising: a camera (FIG 1, imager 11; ¶18) positionable to capture real-time image data (claim 12; ¶19, real-time) of a surgical treatment site (claim 8; ¶¶19-20); an image display (display device 15, ¶20) configured to display the real-time images from the camera (¶19);
at least one processor (claim 11; FIG 3, information processing device 130) and at least one memory storing instructions executable by the at least one processor (claim 21; ¶198) to:
receive the real-time image data (¶81 “Live”) captured within the surgical site (claim 21) and, when available, kinematic information from a robotic component maneuvering a surgical instrument (claim 21; ¶¶51-54);
analyze the image data (¶55, SLAM mapping) to detect and identify at least one surgical instrument at the surgical treatment site (¶¶29-30) and to track motions of the identified surgical instrument (¶29).
Uyama does not expressly teach that the image display displays a plurality of procedural steps for a surgical procedure.
Uyama does not expressly teach determining, based on whether the tracked motions include tool movement patterns or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure, that a predetermined surgical step has been performed or completed; and in response to determining completion of the predetermined surgical steps, automatically update the displayed plurality of procedural steps to record completion of the predetermined surgical step; and automatically extract from the real-time image data a still image captured at the time of completion and store the still image in the at least one memory in a medical record in association with the predetermined surgical step.
Makrinich teaches the image display displays a plurality of procedural steps for a surgical procedure (¶¶169, 171; FIG 9). Makrinich teaches determining (¶203), based on whether the tracked motions include tool movement patterns (¶203) or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure (¶212), that a predetermined surgical step has been performed (¶205) or completed (¶¶212, 219), and in response to determining that the predetermined surgical step has been completed (¶¶171, 212), automatically extract from the real-time image data a still image captured at a time of completion of said predetermined surgical step and store the still image (surgical video frame, ¶213) in the at least one memory in a medical record in association with the predetermined surgical step (¶¶212, 213).
Makrinich also teaches systems comprising at least one processor (¶50) suitable for executing instructions pre-loaded into a memory integrated with or embedded into the controller or stored in a separate memory (¶50). One or more cameras 115 configured to track and identify a surgical tool within an anatomical structure via a computer-based camera control application that uses image recognition algorithms for positioning the camera to capture video/image data of a region-of-interest (ROI) or by a human operator controlling the position of the cameras are taught at ¶61. Surgical video may be recorded in real-time (¶185).
Neither Uyama nor Makrinich teach wherein said plurality of stored tasks or subtasks includes tasks or subtasks designated as ones for while photo documentation is to be captured upon completion thereof, and wherein said predetermined surgical step is a task or subtask so designated; and where the still image is saved independently of any video of the procedure that is stored.
Squilla teaches an automated management information process with surgery-specific preferences 30 (FIG 3). The workflow process shown in FIG 1 is taught as encompassing patient photos and other information “content” related to a medical or surgical procedure (¶17). Content collected within the operating room is taught as including live video, radiological images, and photographs (¶19). FIG 2 illustrates modular information management system 10 that includes the collection, organization, and display of content (¶24). Squilla teaches that system 10 includes controller 12 connected to one or more storage devices 14, one or more content collection devices 16, one or more operator interfaces 18, one or more display devices 24, one or more remote receivers/senders 22 via one or more connection lines 28 (¶25). Content collection devices 16 are configured to retrievably store and/or archive captured content from videos and still images (¶29) and the stored content can be transferred from the content collection devices 16 to the storage device 14 and/or controller 12 during the collection phase of FIG 1 (¶29). Content collection devices 16 can locally store still images and/or videos and can be configured to later upload the substantially live content to the storage device 14 and/or the controller 12 (¶31).
Squilla teaches that the desired content is established (predetermined) at step 30 (FIG 3). Checklists that include all of the heterogeneous content needed (predetermined) are taught as being able to be verified within the “Collection Phase” (FIG 3; ¶41). If no new content is required, the checklist can be determined to be complete and the collected content saved (¶41). The workflow then proceeds to the “Organize Phase” (FIG 4, ¶49). Once those steps have been completed for each selected input in a key phase of a surgical sequence, the associated functionality of that sequence can be saved and the user may proceed to the next phase, the Display Phase (FIG 5; ¶¶50-51).
Squilla teaches universal functionality icons 96 comprising tools configured to access images captured during a surgical procedure (FIGs 6-8, ¶54). Rule set 118 is taught as “governing at least one of the collection phase (FIG. 3), organize phase (FIG. 4), and display phase (FIG. 5) of an exemplary healthcare facility workflow process” and can be based on at least one of many decision factors 120 (FIG 9, ¶69). Decision factors 120 are include doctor-specific preferences 124 comprising the organization of the surgical sequence phases, priority levels, display locations, content desired or requested and determinations regarding content collected and relevancy (FIG 9, ¶72). Content characteristics 122 can include specialist-indicated relevancy determinations which can be utilized as a factor in creating rule set 118 (¶69). Rule set 118 can be utilized to assist in automatically processing a plurality of content (¶77). Hierarchies can be applied to content types including live video and still images (¶84). An automated healthcare facility workflow process can be utilized to suggest to the user a preferred/optimized display layout displaying the plurality of content associated with a surgical procedure (¶84). FIG 9 shows steps by which the content can be organized by automatically associating content-specific functionality with each content of the plurality of content, which can be based on decision factors 120 (¶87).
Accordingly, Squilla teaches the claimed plurality of stored tasks or subtasks (FIGs 1, 3, 6-8, 9, workflow processes; ) includes tasks or subtasks (FIG 9, hierarchical organized content characteristics 122, surgical sequence (as doctor-specific preferences) 124, or specialty/surgical sequence preferences 126, or institution characteristics 128, or medical payer requirements for documentation 129) designated (FIG 9, by rule set 118) as ones for which photo documentation (content collection 102/132; still images ¶¶70, 84) is to be captured upon completion thereof (FIG 9, organized phase 134), and wherein said predetermined surgical step is a task or subtask so designated (FIG 9, rule set 118; claims 1, 9, 12), where the still image is saved independently of any video of the procedure that is stored (“still images”, ¶¶70, 84).
It would have been obvious to one having ordinary skill in the art as of the effective filing date of the invention to combine the teachings of Uyama, Makrinich, and Squilla given that the prior art included each element claimed, although not necessarily in a single reference. Uyama, Makrinich, and Squilla teach in the same field of endeavor, systems comprising processors, memory, and imaging systems for capturing information and making records of surgical procedures.
Although, Uyama discloses the claimed base processor, memory, camera, real-time data capture, and real-time analysis, including capturing and storing a still images, and annotating images while tracking specified positions on the basis of the position/posture information, Uyama does not disclose the functions of determining, based on whether the tracked motions include tool movement patterns or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure, that a predetermined surgical step has been performed or completed and in response to determining that the predetermined surgical step has been completed, automatically extract from the real-time image data a still image captured at a time of completion of said predetermined surgical step and store the still image in the at least one memory in a medical record in association with the predetermined surgical step.
Makrinich specifically addresses algorithmic determinations (¶203), based on whether the tracked motions include tool movement patterns (¶203) or steps in a sequence of movements corresponding to one of a plurality of stored tasks or subtasks for the surgical procedure (¶212), that a predetermined surgical step has been performed (¶205) or completed (¶212) and in response to determining that the predetermined surgical step has been completed (¶212), automatically extract from the real-time image data a still image captured at a time of completion of said predetermined surgical step and store the still image (surgical video frame, ¶213) in the at least one memory in a medical record in association with the predetermined surgical step (¶¶212, 213).
Squilla specifically addresses the claimed plurality of stored tasks or subtasks (FIGs 1, 3, 6-8, 9, workflow processes) includes tasks or subtasks (FIG 9) designated (FIG 9, rule set 118) as ones for which photo documentation (content collection 102/132; still images ¶¶70, 84) is to be captured upon completion thereof (FIG 9, organized phase 134), and wherein said predetermined surgical step is a task or subtask so designated (FIG 9, rule set 118; claims 1, 9, 12), where the still image is saved independently of any video of the procedure that is stored (“still images”, ¶¶70, 84).
Uyama includes the apparatus components of the system required to perform the functions of tool tracking movement patterns, steps or sequences of movements, capturing and storing a still image, annotating images while tracking specified positions on the basis of the position/posture information, and utilizing SLAM mapping, a person of ordinary skill in the art, seeking to utilize Uyama’s hardware architecture and SLAM mapping features for medical record collection, data analytics, and determining that a predetermined step in a surgical plan has been completed based on the imaging data captured during the procedure would reasonably consult Makrinich’s medical record and surgical workflow documentation solution. Makrinich’s use of algorithmic determinations and analytics can be incorporated alongside Uyama’s hardware (same one or more processors, instructions stored in at least one memory executable by the at least one processor, and camera(s) positionable to capture real-time image data of a surgical treatment site), using known methods without redesigning Uyama’s core architecture. Additionally, a person of ordinary skill in the art seeking to utilize Uyama’s hardware architecture and Makrinich’s analytics, could readily incorporate Squilla’s automated healthcare facility workflow processes, including a rules set (FIG 9, 118) that can be assigned and determined by any or all of the decision factors 120 (FIG 9) which can then automatically process content (collect, organize, and display/save/archive/collaborate) in order to automate the image capturing process of multiple individuals and groups in one workflow system. One of ordinary skill in the art would be motivated to add Squilla’s automated solution because it is modular, encompasses automatic processing of a plurality of content from a heterogenous content sources automating the workflow based on a rule set comprising hierarchical organization which encompasses still images that are displayed, saved, and archived.
Because the references address the same engineering problem (documenting real-time surgical imaging using at least one processor, at least one memory, and a camera positionable to capture real-time image data of a surgical treatment site) and the proposed modifications are mechanically compatible and implemented by routine engineering practices (adding software functions based on tracked motions including tool movement patterns or sequential surgical procedure steps, documenting that the predetermined surgical step has been completed and automatically extracting a still image (frame) from the real-time image data captured at a time of completion of a surgical step and storing the still images (frames) in the at least one memory in a medical record in association with the predetermined surgical step, where the steps are part of a healthcare facility workflow process that is automated and designed to utilize a rules-based platform for plural content from heterogeneous content sources), a person of ordinary skill in the art before the effective filing date of the claimed invention would have had a reasonable expectation of success in combining these teachings because they all work together as demonstrated by Squilla.
One of ordinary skill in the art recognizes that computer-implemented processes are results-effective variables which can be optimized when using a general computer. The system encompasses both hardware and software. One of ordinary skill in the art would have had a reasonable expectation of success to modify software with workflow processes based on the modified rejections provided below in response to Applicant’s arguments and amendments. Such processes can be optimized by a person of ordinary skill in the art without undue experimentation based on the teachings of the references as set forth below. The references themselves provide the teaching, suggestion, and motivation for computer-implemented workflow processes in a healthcare setting where modifications thereof would amount to nothing more than routine experimentation that can be optimized on an individual use case basis. See, In re Antonie, 559 F.2d 618, 195 USPQ 6 (CCPA 1977) and In re Boesch, 617 F.2d 272, 205 USPQ 215 (CCPA 1980)).
Regarding claim 6, Uyama modified by Makrinich and Squilla teaches the system of claim 1, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the instructions are executable by said at least one processor (¶50) to determine completion of the predetermined step based on detected movement patterns of the surgical tool (¶219). Squilla teaches that system 10 may keep track of changes made to preferences and rules to assist in an automated content request process, after performing multiple workflow processes (FIG 9, ¶79).
Regarding claim 7, Uyama modified by Makrinich and Squilla teaches the system of claim 1, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the instructions are executable by said at least one processor (¶50) to determine completion of the predetermined step based on detected sequences of movements of the surgical tool (¶219).
Squilla teaches that system 10 may keep track of changes made to preferences and rules to assist in an automated content request process, after performing multiple workflow processes (FIG 9, ¶79).
Regarding currently amended claim 8, Uyama modified by Makrinich and Squilla teaches the system of claim 1, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the instructions are executable by said at least one processor (¶50) to analyze the image data to detect anatomical features at the surgical treatment site (¶148) and to detect changes in the detected anatomical features (¶210), and to determine completion of the predetermined step based on detected changes to the anatomical feature (¶¶149, 212, 219).
Squilla teaches that system 10 may keep track of changes made to preferences and rules to assist in an automated content request process, after performing multiple workflow processes (FIG 9, ¶79).
Regarding claim 9, Uyama modified by Makrinich and Squilla teaches the system of claim 8, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the detected changes include detected changes in position of the anatomical feature (¶¶156, 157).
Regarding claim 10, Uyama modified by Makrinich and Squilla teaches the system of claim 8, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the detected changes include detected changes in shape of the anatomical feature (dimensions of anatomical structures, ¶156).
Regarding claim 12, Uyama modified by Makrinich and Squilla teaches the system of claim 1, as set forth above, for the reasons set forth above.
Squilla teaches wherein the instructions are executable by said at least one processor (FIG 9, by rule set 118) to create a medical record storing the still image (content collection 102/132; still images ¶¶70, 84).
Regarding claim 13, Uyama modified by Makrinich and Squilla teaches the system of claim 12, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the system includes a voice input device (audio sensors 425; ¶138), and wherein the instructions are executable by said at least one processor (¶50) to, upon detecting completion of the predetermined step, prompt a user to speak an annotation (¶93), in response to the user speaking the annotation (¶93), receiving data corresponding to the annotation from the voice input device (¶138), store the annotation in the medical record in association with the still image (surgical video frame, ¶213).
Regarding claim 14, Uyama modified by Makrinich and Squilla teaches the system of claim 1, as set forth above, for the reasons set forth above.
Squilla teaches wherein the still image (content collection 102/132; still images ¶¶70, 84) shows completion of the predetermined step at the surgical treatment site (FIG 9 by rule set 118).
Regarding claim 15, Uyama modified by Makrinich and Squilla teaches the system of claim 4, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the instructions are executable by said at least one processor (¶50) determine completion of the predetermined step based on detected movement patterns of the surgical tool (¶219).
Regarding claim 16, Uyama modified by Makrinich and Squilla teaches the system of claim 4, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the instructions are executable by said at least one processor (¶50) to determine completion of the predetermined step based on detected sequences of movements of the surgical tool (¶219). Squilla teaches that system 10 may keep track of changes made to preferences and rules to assist in an automated content request process, after performing multiple workflow processes (FIG 9, ¶79).
Regarding claim 17, Uyama modified by Makrinich and Squilla teaches the system of claim 4, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the instructions are executable by said at least one processor (¶50) to analyze the image data to detect anatomical features at the surgical treatment site (¶148) and to detect changes in the detected anatomical features (¶210), and to determine completion of the predetermined step based on detected changes to the anatomical feature (¶¶149, 212, 219).
Regarding claim 18, Uyama modified by Makrinich and Squilla teaches the system of claim 17, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the detected changes include detected changes in position of the anatomical feature (¶¶156, 157).
Regarding claim 19, Uyama modified by Makrinich and Squilla teaches the system of claim 17, as set forth above, for the reasons set forth above.
Makrinich teaches wherein the detected changes include detected changes in shape of the anatomical feature (dimensions of anatomical structures, ¶156).
Regarding claim 21, Uyama modified by Makrinich and Squilla teaches the system of claim 4, as set forth above, for the reasons set forth above.
Squilla teaches wherein the still image (content collection 102/132; still images ¶¶70, 84) shows completion of the predetermined step at the surgical treatment site (FIG 9, by rule set 118).
Conclusion
No claim is allowed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHERIE M POLAND whose telephone number is (703)756-1341. The examiner can normally be reached M-F 9am-6pm (CST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jackie Ho can be reached at 571-272-4696. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHERIE M POLAND/Examiner, Art Unit 3771
