Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 9, 2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s arguments, filed 5/26/2026, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 101 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Olmeda Reino et. al. (United States Patent Application Publication US 2020/0218904 A1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4, 7, 10, 13, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Olmeda Reino et. al. (United States Patent Application Publication US 2020/0218904 A1) in view of Suchan, Jakob, et al. "Visual explanation by high-level abduction: On answer-set programming driven reasoning about moving objects." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018. (Year: 2018), and Johnson et. al. (United States Patent US 12,106,654 B2).
Regarding claim 1, Olmeda Reino et. al. discloses a method for tracking movement, comprising: performing, with a neural network model, person detection in frames from multiple video streams generated by video cameras (Olmeda Reino et. al. [0021]: a system for tracking objects in a temporal sequence of digital images. The system is configured to: detect potential objects in the images, the detected potential objects being indicated as nodes, identify pairs of neighboring nodes, such that for each pair the nodes of said pair potentially represent an identical object and their spatial and/or temporal relationship with each other is within a predetermined range, connect each pair of neighboring nodes with a first type edge, identify at least one supplementary pair of distance nodes whose spatial and/or temporal relationship with each other exceeds the predetermined range); combining visual and location information from the detection images with the neural network model that determines scores for pairs of detection images across the multiple video streams and across frames of respective video streams based on a similarity between visual features and location coordinates of the pairs of detection images (Olmeda Reino et. al. [0021]: a system for tracking objects in a temporal sequence of digital images. The system is configured to: detect potential objects in the images, the detected potential objects being indicated as nodes, identify pairs of neighboring nodes, such that for each pair the nodes of said pair potentially represent an identical object and their spatial and/or temporal relationship with each other is within a predetermined range, connect each pair of neighboring nodes with a first type edge, identify at least one supplementary pair of distance nodes whose spatial and/or temporal relationship with each other exceeds the predetermined range, connect the pair of distance nodes with a supplementary second type edge, each of the first and second type edges being assigned a cost value representing a probability whether the connected nodes represent an identical object or not; [0023]: An object (e.g. a human) may be detected in several images so that the resulting nodes have a temporal offset and eventually also a spatial offset to each other. [0024]: The identified pairs of neighboring nodes, which are each connected by a first type edge, fulfill two conditions: The nodes of a pair potentially represent an identical (i.e. same) object. In addition, the spatial and/or temporal relationship of the connected neighboring nodes is within a predetermined range. [0031]); generating a pairwise detection graph with the neural network model using the detection images as nodes and the scores as weighted edges (Olmeda Reino et. al. [0040]-[0047]: CNN neural network is used to detect the potential objects in the images, Fig. 2a-2d; [0030]: multi-person tracking is cast as a minimum cost multicut problem. There and in the present disclosure, every detection (i.e. detected object) is represented by a node in a graph; edges connect detections within and across time frames, and costs assigned to edges can be positive, to encourage the incident nodes to be in the same track, or negative, to encourage the incident nodes to be in distinct tracks.).
PNG
media_image1.png
700
658
media_image1.png
Greyscale
Olmeda Reino et. al. fails to disclose video cameras within a healthcare facility to identify detection images; tracking movement of an individual with the neural network model based on a constrained answer set programming problem that determines whether entity detections for the individual belong to a same track that includes different views and frames of the entity detections, with constraints determined based on matching scores and logical assumptions; and performing, with the neural network model, an action responsive to the tracked movement of the individual within the healthcare facility by switching camera views that overcomes occlusions within the camera views.
Suchan teaches tracking movement of an individual based a constrained answer set programming problem (Suchan, A Hybrid Architecture for Visual Explanation based on the integration of high-level abductive reasoning within Answer Set Programming (ASP)), with constraints determined based on matching scores and logical assumptions (Suchan, Figure 2 People Movement, Beliefs as (Spatial) constraints).
PNG
media_image2.png
436
494
media_image2.png
Greyscale
PNG
media_image3.png
340
948
media_image3.png
Greyscale
Johnson et. al teaches a method for tracking movement in a healthcare facility, comprising: performing person detection in frames from multiple video streams in a healthcare facility to identify detection images (Abstract, claim 9)
PNG
media_image4.png
506
504
media_image4.png
Greyscale
PNG
media_image5.png
260
356
media_image5.png
Greyscale
and performing, with the neural network model, an action responsive to the tracked movement of the individual within the healthcare facility by switching camera views that overcomes occlusions within the camera views (Johnson et. al: Col. 7, lines 62-67, Col. 8, lines 5-
23).
PNG
media_image6.png
84
320
media_image6.png
Greyscale
PNG
media_image7.png
330
324
media_image7.png
Greyscale
Suchan is analogous to the claimed invention because it addresses the answer set programming problem for visual perception and object tracking of moving people in the setting of movies.
Johnson is analogous to the claimed invention because it is pertinent to the problem of patient monitoring in a healthcare facility for fall reduction. One of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to incorporate the teaching of Johnson for the benefit of fall mitigation of tracked patients in a healthcare facility.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method for video tracking of Olmeda Reino et. al. to incorporate the teachings of Suchan and Johnson et. al. so that the solution of the claimed invention is fully addressed and to have a more robust and accurate person image feature representation.
Regarding claim 4, the combination of Olmeda Reino et. al., Suchan et. al., and Johnson et. al. discloses the method of claim 1, and Suchan further discloses further comprising extracting the location information based on a projection of two-dimensional coordinates into a three-dimensional environment for the detection images and determining a distance between the projected coordinates (Suchan, Ontology: Space, Time, Objects, Events where the tracks of the objects are measured in basic spatial 2D and 3D space coordinates).
PNG
media_image8.png
502
496
media_image8.png
Greyscale
PNG
media_image9.png
746
492
media_image9.png
Greyscale
It is important to the claimed invention to have quantified distance between 2D coordinates in a 3D environment. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have included the teachings of Suchan with the teachings of Olmeda Reino et. al. and Johnson et. al. so that the distance between two defined coordinates between people in a scene is measured and recorded.
Regarding claim 7, the combination of Olmeda Reino et. al., Suchan et. al., and Johnson et. al. discloses the method of claim 1, and Johnson et. al. further teaches wherein the action includes securing access to areas within the healthcare facility to hinder movement of the individual to the areas (Johnson et. al: Col. 7, lines 62-67, Col. 8, lines 5-23: a healthcare provider can take action to secure access to certain areas.)
Regarding claim 10, which is a system for tracking movement, comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to: perform the method of claim 1, which the rejection analysis of claim 1 is incorporated herein.
Regarding claim 13, the combination of Olmeda Reino et. al., Suchan et. al., and Johnson et. al. further discloses the system of claim 10, wherein the computer program further causes the hardware processor to extract the location information based on a projection of two-dimensional coordinates into a three-dimensional environment for the detection images and determining a distance between the projected coordinates (Suchan, Ontology: Space, Time, Objects, Events where the tracks of the objects are measured in basic spatial 2D and 3D space coordinates).
It is important to the claimed invention to have quantified distance between 2D coordinates in a 3D environment. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have included the teachings of Suchan with the teachings of Wu, Shen, and Buehler et. al. so that the distance between two defined coordinates between people in a scene is measured and recorded.
Regarding claim 19, which overlaps substantially in scope as in claim 1 except for “a method for tracking movement in a healthcare facility”, and “generating a report for a healthcare
professional for decision-making related to a patient’s treatment, based on the tracked movement”. Thus, the rejection analysis of claim 1 is incorporated herein. Johnson et. al. teaches and generating a report for a healthcare professional for decision-making related to a patient’s treatment, based on the tracked movement (Johnson et. al: Col. 7, lines 62-67, Col. 8, lines 5-23).
Claim(s) 2, 3, 5, 11-12. 14-17, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Olmeda Reino et. al. (United States Patent Application Publication US 2020/0218904 A1) in view of Suchan, Jakob, et al. "Visual explanation by high-level abduction: On answer-set programming driven reasoning about moving objects." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018. (Year: 2018), and Johnson et. al. (United States Patent US 12,106,654 B2) as applied to claim 1 above, and further in view of Buehler et. al. (European Patent EP-1563686-B1).
Regarding claim 2, the combination of Olmeda Reino et. al., Suchan et. al., and Johnson et. al. discloses the method of claim 1. The combined teachings fail to teach the following limitations as further claimed. Buehler et. al. teaches further comprising synchronizing the multiple video streams to identify temporal correspondences between frames of the multiple video streams (Buehler et. al. Figures 1, 5). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have included the teachings of Buehler et. al. with the teachings of Olmeda Reino et. al., Suchan, and Johnson et. al. so that the multiple video streams are observed for a substantially long period of recorded time.
PNG
media_image10.png
750
574
media_image10.png
Greyscale
.
Regarding claim 3, the combination of Olmeda Reino et. al., Suchan et. al., and Johnson et. al. discloses the method of claim 1. Buehler et. al. further teaches further comprising extracting the visual information based on a visual similarity between detection images (Buehler et. al. Figures 6A, 6B, 7A, 7B). It is important to the claimed invention to have identified a target of visual similarity that defines the human of interest. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have included the teachings of Buehler et. al. with the combination of Olmeda Reino et. al., Suchan et. al., and Johnson et. al. so that there is an objective target identified.
Regarding claim 5, the combination of Olmeda Reino et. al., Suchan et. al., and Johnson et. al. discloses the method of claim 1. Buehler et. al. further teaches wherein generating the pairwise detection graph includes determining edges between detection images from different frames of a same video stream and determining edges between detection images from different video streams at corresponding times (Buehler et. al. Figures 6A, 6B, 7A, 7B). It is critical to the claimed invention to produce a graph that is from pairwise detection results so that the identification of people can be visualized. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have included the teachings of Buehler et. al. so that human surveillance solutions can be fully utilized.
Regarding claim 11, the combination of Olmeda Reino et. al., Suchan, and Johnson et. al. teaches the system of claim 10. Buehler et. al. further teaches wherein the computer program further causes the hardware processor to synchronize the multiple video streams to identify temporal correspondences between frames of the multiple video streams (Buehler et. al. Figures 7A, 7B). It is important to the claimed invention to have the complete computer program and hardware necessary to carry out the tasks disclosed. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have included the teachings of Buehler et. al. that also includes a system of CCTV surveillance for security purposes.
Regarding claim 12, the combination of Olmeda Reino et. al., Suchan, and Johnson et. al. teaches the system of claim 10. Buehler et. al. further teaches wherein the computer program further causes the hardware processor to extract the visual information based on a visual similarity between detection images (Buehler et. al. [0089]).
PNG
media_image11.png
22
736
media_image11.png
Greyscale
PNG
media_image12.png
184
738
media_image12.png
Greyscale
It is important to the claimed invention to have the complete computer program and hardware necessary to carry out the tasks disclosed. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have included the teachings of Buehler et. al. that also includes a system of CCTV surveillance for security purposes.
Regarding claim 14, the combination of Olmeda Reino et. al., Suchan, and Johnson et. al. teaches the system of claim 10. Buehler et. al. further teaches wherein the computer program further causes the hardware processor to determine edges between detection images from different frames of a same video stream and to determine edges between detection images from different video streams at corresponding times (Buehler Figure 4).
PNG
media_image13.png
863
339
media_image13.png
Greyscale
The edge detection of detection images is an important solution to the claimed invention because it reduces error in overlapping video streams with multiple people. Thus, it would have been obvious to one skilled in the art prior to the effective filing date to have included the teachings of Buehler et. al. with the teachings of Olmeda Reino et. al., Suchan, and Johnson et. al. to include the solution for edge detection in multiple video frames.
Regarding claim 17, the combination of Olmeda Reino et. al., Suchan, and Johnson et. al. teaches the system of claim 10. Buehler et. al. further teaches wherein the computer program further causes the hardware processor to an output from a visual branch to an output of a location branch to combine the visual and location information (Buehler et. al. Figure 15, Figure 4, Figure 1).
PNG
media_image14.png
696
640
media_image14.png
Greyscale
This is an important aspect for person identification based on time and location, especially related to personal security. Thus, it would have been obvious to one skilled in the art prior to the effective filing date to have included the teachings of Buehler et. al. so that the information collected is matched to the correct person of interest.
Regarding claim 20, Buehler et. al. in the combination further teaches the method of claim 19, wherein generating the pairwise detection graph includes determining edges between detection images from different frames of a same video stream (Buehler et. al. Fig. 4, 5) and determining edges between detection images from different video streams at corresponding times (Buehler et. al. Fig. 4, [0057]).
PNG
media_image15.png
152
728
media_image15.png
Greyscale
Regarding claims 15, 16, the rejection of claims 19-20 above is fully incorporated herein.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Olmeda Reino et. al. (United States Patent Application Publication US 2020/0218904 A1) in view of Suchan, Jakob, et al. "Visual explanation by high-level abduction: On answer-set programming driven reasoning about moving objects." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018. (Year: 2018), and Johnson et. al. (United States Patent US 12,106,654 B2) as applied to claim 1 above, and further in view of Ross (United States Patent US 2022/0022006A1.
Regarding claim 6, Olmeda Reino et. al., Suchan et. al., and Johnson et. al. disclose the method of claim 1. The combined teachings fail to teach the following limitations as further recited. Ross et. al. further teaches wherein the action includes generating a report for a healthcare professional for decision-making related to a patient's treatment, based on tracked movement of the patient (Ross Fig. 1, 3, Abstract).
PNG
media_image16.png
616
560
media_image16.png
Greyscale
PNG
media_image17.png
726
946
media_image17.png
Greyscale
Ross is considered analogous to the claimed invention because it is reasonably
pertinent to the problem of providing safety and security for patients receiving treatment and will
allow healthcare professionals to make an informed treatment decision based on recorded
movement data. It would have been obvious to a person having ordinary skill in the art before
the effective filing date of the claimed invention to incorporate the teaching of Ross for the
benefit of generating a report for a healthcare professional for decision-making related to a
patient’s treatment, based on tracked movement of the patient (Ross Fig. 1, 3, Abstract).
Claim(s) 8-9, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Olmeda Reino et. al. (United States Patent Application Publication US 2020/0218904 A1) in view of Suchan, Jakob, et al. "Visual explanation by high-level abduction: On answer-set programming driven reasoning about moving objects." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018. (Year: 2018), and Johnson et. al. (United States Patent US 12,106,654 B2) as applied to claim 1, and further in view of Shen, Yantao, et al. "Person re-identification with deep similarity-guided graph neural network." Proceedings of the European conference on computer vision (ECCV). 2018. (Year: 2018).
Regarding claim 8, the combination of Olmeda Reino et. al., Suchan, and Johnson et. al. discloses the method of claim 1. The combined teachings fail to teach the following limitations as further recited. Shen further teaches wherein combining the visual and location information includes adding an output from a visual branch to an output of a location branch (Shen, 2.2 (Graph for Machine Learning)—“After the message propagation among different nodes (samples), the mapping function will output the classification or regression results of each node”).
PNG
media_image18.png
376
558
media_image18.png
Greyscale
Shen et. al. is considered analogous to the claimed invention because it is reasonably pertinent to the problem of person reidentification, which aims at finding the person in images of interest in a set of images across different cameras without error in the intelligent surveillance systems. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method for video tracking to incorporate the teachings of Shen et.al by including the re-identification solution of having a more robust and accurate person image feature representation through fusion weights for updating the nodes’ features (Shen et. al 2.2 (Graph for machine learning)).
PNG
media_image19.png
120
558
media_image19.png
Greyscale
PNG
media_image20.png
182
572
media_image20.png
Greyscale
Regarding claim 9, Shen in the combination also teaches the method of claim 8, wherein the visual branch includes processing the detection images with a re-identification model (Shen et. al. Abstract).
Regarding 18, the rejection of claim 9 is incorporated herein.
Conclusion
Response to Amendment
Examiner has carefully considered the amendments to the claims 1-20. However, after an updated search, new prior art was found to reject all the claims with new grounds of rejection.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA YIFANG LIN whose telephone number is (571)272-6435. The examiner can normally be reached M-F 7:00am-6:15pm, with optional day off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSICA YIFANG LIN/Examiner, Art Unit 2668 July 10, 2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights