DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
All amendments to the claims as filed on 12/10/2025 have been entered and the action follows:
Response to Arguments
Applicant’s arguments with respect to claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10, 17-24 and 33-34 are rejected under 35 U.S.C. 103 as being unpatentable over Grantcharov (US Pub. 2017/0249432) in view of Singh et al (US Pub. 2022/0036109).
With respect to claim 1, Grantcharov discloses A computer-implemented method for traffic monitoring in an operating room (see paragraph 0132, wherein …measuring foot traffic by recording the opening and closing of doors and number of individuals present in the room), the method comprising:
receiving video data of an operating room, the video data captured by a camera having a field of view for viewing movement of a plurality of individuals in the operating room during a medical procedure [each frame of the video data being used in inference to generate bounding box annotations for each frame of the video data to at least one body part visible in each frame of the video data ], (see paragraph 0132, wherein …two wide-angle cameras …installed to capture data representative of an entire view (e.g. 180 degree or more) of the room…Both entrances to the room may be in the field of view, which allows for measuring foot traffic by recording the opening and closing of doors and number of individuals present in the room);
storing an event data model including data defining a plurality of possible events within the operating room, the event data model trained to utilize a total number of people in the operating room as a conditioning variable, and trained using a data set of training frames from training procedures used to update a set of weight parameters representing a trained event data model, (see figure 2, numerical 44 various models; and paragraph 0125, wherein …The multi-channel recording device 40 may synchronize and record the feeds to generate output data 44 (e.g. for export as a session file). The output data may include, for example, measurement values to assess individual and team performance, identify errors and adverse events and link to outcomes, evaluate performance and safety of technology, and assess efficiency, and paragraph 0224 wherein …The present solution may apply CI methodologies, …to develop robust networks and models that will extract features, detect correlations, and identify patterns of events from the OR black box dataset);
processing the video data using one or more [trained object-tracking] models configured to track movement of objects within the operating room, the objects [including at least one body part], and the processing using [at least one detector trained] to detect a given type of the objects, the movement of the objects processed to estimate changes to a total number of people in the operating room based on the at least one body part in the video data at a particular time, [the at least one body part in the video data based on the bounding box annotations around each of the heads of the people in the operating room across a batch of a plurality of frames], and to record a timestamp when the estimated change occurs, [and to record a data output containing the bounding box annotations and a corresponding confidence score associated with each of the bounding box annotations], (see paragraph 0195, wherein …The black-box recording device or encoder may provide for analysis of technical and non-technical individual and team performance, errors, event patterns, risks and performance of medical/surgical devices in the OR/patient intervention areas; see paragraph 0222, wherein …instrument may operate by monitoring the current conditions in the OR, reporting events that may lead to conditions of potential errors (e.g., the noise level, temperature, number of individuals in the room, and so on)...; see paragraph 0174, wherein …feature of the Black Box Encoder Analytics unit may be its relational database that captures and cross-references the entire dataset composition which includes, but is not limited to: the complete resultant annotated and tag content streams produced by the Rich Content Analysis software identified with structured meta-data such as the Technical Procedural Rating System for Laparoscopic Bypass, and so on; facility variables such as …number of medical staff present and what their designation is, and so on; procedure case notes (in a structured well-formed relational data model) such as what kind of stapler was used, was hemostatic agent used, and so on…; see paragraph 0204, wherein …Smart devices or smart adaptors will process …the captured media and data, and embed a timestamp marker at precise timeline intervals in the output file...);
at a particular time or duration of time, determining a likelihood of occurrence of one of the possible events based on the tracked movement and by processing the estimated total number of people in the operating room as the conditioning variable against the trained event data model, [the likelihood of occurrence determined using a head count model configured to determine a most probable new location of the at least one body part in the video data based on the bounding box annotations across a plurality of frames and the corresponding confidence scores by performing geometric and pixel transformations such that even the at least one body part that are partially obstructed are counted despite an obstruction]; and
generating a data output representative of the likelihood of occurrence of one of the possible events, [the data output including at least a first data output recording changes to the number of people in the operating room, and a second data output containing the bounding box annotations and a confidence score associated with each of the bounding box annotations representative of a corresponding confidence score], (see paragraph 0222, wherein …Such an instrument may operate by monitoring the current conditions “a particular time” in the OR, reporting events “a data output” that may lead to conditions of potential errors (e.g., the noise level, temperature, number of individuals in the room “number of people in the operating room”, and so on…), as claimed.
However, Grantcharov fails to explicitly disclose receiving video data of an operating room, the video data captured by a camera having a field of view for viewing movement of a plurality of individuals in the operating room during a medical procedure [each frame of the video data being used in inference to generate bounding box annotations for each frame of the video data to at least one body part visible in each frame of the video data];
processing the video data using one or more [trained object-tracking] models configured to track movement of objects within the operating room, the objects [including at least one body part], and the processing using [at least one detector trained] to detect a given type of the objects, the movement of the objects processed to estimate changes to a total number of people in the operating room based on the at least one body part in the video data at a particular time, [the at least one body part in the video data based on the bounding box annotations around each of the heads of the people in the operating room across a batch of a plurality of frames], and to record a timestamp when the estimated change occurs, [and to record a data output containing the bounding box annotations and a corresponding confidence score associated with each of the bounding box annotations];
at a particular time or duration of time, determining a likelihood of occurrence of one of the possible events based on the tracked movement and by processing the estimated total number of people in the operating room as the conditioning variable against the trained event data model, [the likelihood of occurrence determined using a head count model configured to determine a most probable new location of the at least one body part in the video data based on the bounding box annotations across a plurality of frames and the corresponding confidence scores by performing geometric and pixel transformations such that even the at least one body part that are partially obstructed are counted despite an obstruction]; and
generating a data output representative of the likelihood of occurrence of one of the possible events, [the data output including at least a first data output recording changes to the number of people in the operating room, and a second data output containing the bounding box annotations and a confidence score associated with each of the bounding box annotations representative of a corresponding confidence score], (emphasis added) as claimed.
Singh teaches receiving video data of an operating room, the video data captured by a camera having a field of view for viewing movement of a plurality of individuals in the operating room during a medical procedure each frame of the video data being used in inference to generate bounding box annotations for each frame of the video data to at least one body part visible in each frame of the video data, (see figure 2, numerical 210 the boundary box and paragraph 0070);
processing the video data using one or more [trained object-tracking] models configured to track movement of objects within the operating room, the objects [including at least one body part], and the processing using [at least one detector trained] to detect a given type of the objects, the movement of the objects processed to estimate changes to a total number of people in the operating room based on the at least one body part in the video data at a particular time, the at least one body part in the video data based on the bounding box annotations around each of the heads of the people in the operating room across a batch of a plurality of frames, (see figure 1, numerical 130 Detection and numerical 140 Tracking and counting; and figure 2, numerical 210 boundary box around body part of the person), and to record a timestamp when the estimated change occurs, and to record a data output containing the bounding box annotations and a corresponding confidence score associated with each of the bounding box annotations, (see figure 5 numerical 524 memory this is use to record or store any data];
at a particular time or duration of time, determining a likelihood of occurrence of one of the possible events based on the tracked movement and by processing the estimated total number of people in the operating room as the conditioning variable against the trained event data model, the likelihood of occurrence determined using a head count model configured to determine a most probable new location of the at least one body part in the video data based on the bounding box annotations across a plurality of frames and the corresponding confidence scores by performing geometric and pixel transformations such that even the at least one body part that are partially obstructed are counted despite an obstruction, (see paragraph 0030 wherein …The one or more images may include multiple features, such as foreground, edges, motion, and occlusion features. The pre-processing block 110 extracts the features from the images. A first pre-processing engine in the pre-processing block 110 extracts motion features and occlusion features from the images. Occlusion features are defined as pixel-level features where occluding foreground pixels are extracted…); and
generating a data output representative of the likelihood of occurrence of one of the possible events, the data output including at least a first data output recording changes to the number of people in the operating room, and a second data output containing the bounding box annotations and a confidence score associated with each of the bounding box annotations representative of a corresponding confidence score, (see figure 5 numerical 510 display device], (emphasis added) as claimed.
It would have been obvious to one ordinary skilled in the art at the effective date of invention to combine the two references as they are analogous because they are solving similar problem of person detection and tracking using image analysis. The teaching of Singh to train a model to detect and count the person using boundary box “bounding box” can be incorporated in to Grantcharov as suggested (see Grantcharov paragraph 0222, …neural network…), for suggestion, and modifying the system yields a tracking a person(s) in a area, (see Singh paragraph 0003), for motivation.
With respect to claim 2, combination of Grantcharov and Singh further discloses wherein the at least one body part includes at least one of a limb, a hand, a head, or a torso, (see Singh figure 2, numerical 210 with hand in the boundary box), as claimed.
With respect to claim 3, combination of Grantcharov and Singh further discloses wherein the plurality of possible events includes adverse events, (see Grantcharov paragraph 0022, wherein …Embodiments described herein …which provides comprehensive data collection …such settings to: identify and/or analyze errors, adverse events and/or adverse outcomes …), as claimed.
With respect to claim 4, combination of Grantcharov and Singh further discloses wherein the Head count model is configured with an adjustable learning rate to avoid local extrema, (see Singh paragraph 0032, wherein … This can be achieved with a selective learning “adjustable learning rate” of the background, with masks around people such that they are not learned as background…), as claimed.
With respect to claim 5, combination of Grantcharov and Singh further discloses wherein determining a the likelihood of occurrence of one of the possible events includes determining that the count of people exceeds a pre-defined threshold, (see Grantcharov paragraph 0132, wherein …Both entrances to the room may be in the field of view, which allows for measuring foot traffic by recording the opening and closing of doors and number of individuals present in the room, ) as claimed.
With respect to claim 6, combination of Grantcharov and Singh further discloses a separate object-tracking model of a plurality of object-tracking models is utilized to detect different types of objects in the operating room, (see Grantcharov paragraph 0174, wherein …Black Box Encoder Analytics unit may be its relational database that captures and cross-references the entire dataset composition which includes, but is not limited to…facility variables such as Department, Operating Room “detect different types of objects in the operating room” and so on; …number of medical staff present and what their designation is, and so on…), as claimed.
With respect to claim 7, combination of Grantcharov and Singh further discloses wherein the count describes a number of individuals in a portion of the operating room, the portion of the operating room defined using a stored floorplan data structure including data and metadata that describes a floorplan or layout of at least a portion of the operating room such that positions of objects are estimated with reference to a three-dimensional coordinate system relative to the operating room, and wherein the determination of the likelihood of occurrence of one of the possible events based on the tracked movement includes providing the estimated positions of objects to the trained event data model, (see Grantcharov paragraph 0192, wherein …FIGS. 13 to 15 illustrate schematics of various example views according to some embodiments. For example, FIG. 13 illustrates a schematic interface with a graphical indicator 150 of display data feeds and a graphical indicator of an OR layout with example positioning of various data capture devices; and paragraph 0215, wherein …platform may implement motion tracking techniques using various components and data transformations. For example, the platform may include one or more autonomous or semi-autonomous 3D depth cameras or Time-of-Flight (TOF) sensors using laser and/or infra-red (IR) devices. As another example, the platform may generate distance and/or position information from the output signal of the TOF sensor and that it converts into a 3D depth map or point cloud. Embodiments described herein may include a computing device for processing output data from 3D camera or TOF sensor. Embodiments described herein may provide customized data processes to distinguish motion resulting from changes in captured depth maps. Embodiments described herein may provide media management hardware and software to aggregate, package, compress, encrypt and synchronize captured point clouds as motion data with other collected media…), as claimed.
With respect to claim 8, combination of Grantcharov and Singh further discloses determining a correlation between the likely occurrence of one of the possible events and a distraction, (see Grantcharov paragraph 0098, wherein …According to some embodiments, this information then may be synchronized …and/or used to evaluate: technical performance of the healthcare providers; non-technical performance of the clinical team members; patient safety (through number of registered errors and/or adverse events); occupational safety; workflow; visual and/or noise distractions; and/or interaction between medical/surgical devices and/or healthcare professionals, etc), as claimed.
With respect to claim 9, combination of Grantcharov and Singh further discloses wherein the one or more object-tracking models are trained the data set of training frames from training procedures used to update the set of weight parameters representing the trained event data model include variations with at least one of occlusions, different colored caps, and masks, and wherein the trained model and its weights are exported to an inference graph data object, (see Grantcharov paragraph 0222, wherein …the objective is to use Computational Intelligence (CI) to reconstruct a mathematical model that recognizes key factors and predicts clinical outcomes, costs and safety hazards. CI tools may include neural networks…; this obviate that a training is done using weights/coefficients to detect any object/person i.e. “…of training frames from training procedures used to update the set of weight parameters representing the trained event data model include variations with at least one of occlusions, different colored caps, and masks, and wherein the trained model and its weights are exported to a inference graph data object”), as claimed.
With respect to claim 10, combination of Grantcharov and Singh further discloses wherein the object-tracing models are trained to track a proximity of an object relative to a radiation-emitting device, (see Grantcharov paragraph 0222, wherein …the objective is to use Computational Intelligence (CI) to reconstruct a mathematical model that recognizes key
factors and predicts clinical outcomes, costs and safety hazards. CI tools may include neural networks…; this obviate that a training is done using weights/coefficients to detect any object/person i.e. “…are trained to track a proximity of an object…”), as claimed.
Claims 17-24 and 33-34 are rejected for the same reasons as set forth for the rejections for claims 1-10, because claims 17-24 and 33-34 are claiming subject matter of similar scope of various combinations as claimed in claims 1-9.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VIKKRAM BALI whose telephone number is (571)272-7415. The examiner can normally be reached Monday-Friday 7:00AM-3:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VIKKRAM BALI/Primary Examiner, Art Unit 2663