DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting: Note
Claims 1-10 are not currently rejected on the ground of nonstatutory double patenting as being unpatentable over US 12148173 B2 given there appear to be sufficient differences between the wording of the claims at present. However, applicant is advised to be mindful of language similarities in any future claim amendments going forward.
Drawings
The drawings were received on 23 October, 2024. These drawings are accepted.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites an object tracking method performed by a computer and comprising: receiving a request for specifying a first object; determining the first object captured in a video as an object to be tracked; associating the first object with a second object having a predetermined relationship with the first object; and tracking the first object in a case where the first object and the second object are detected in the video.
The limitation of receiving a request is interpreted as insignificant pre-solution activity as there is no description of how the request is being actively performed. The limitation of determining, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting a computer, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the computer language, “determining” in the context of this claim encompasses the user manually determining there is a person in the video. Similarly, the limitation of associating, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. For example, but for the computer language, “associating” in the context of this claim encompasses the user determining if a person in the video is interacting with a car or bag. Similarly, the “tracking” can encompass a person seeing if the person and baggage are still there in subsequent frames. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim only recites one additional element – using a computer to perform the determining, associating, and tracking steps. The computer is recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of determining and tracking) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a computer to perform steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 2 just defines further tracking conditions.
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 3 just defines further tracking relationships.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 4 just defines further tracking relationships
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 5 just describes an order of operations.
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 6 just defines what is being tracked.
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 7 just defines what is being tracked.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 8 just defines what is being tracked.
Claims 9 and 10 are rejected under 35 U.S.C. 101 for the same reasons as claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, and 6-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Strimling (US 20140334684 A1) in view of Manasseh et al. (US 20050128304 A1).
Regarding claims 1, 9, and 10, Strimling discloses an object tracking method performed by a computer (A system and method may include a station to identify a vehicle and to determine a characterizing feature of the vehicle. An imager that is located remotely from the station is configured to acquire an image of the vehicle that includes the characterizing feature. A processor is configured to detect the characterizing feature in the acquired image and to associate the detected characterizing feature with the identified vehicle so as to predict a location of the vehicle, abstract) and comprising; an object tracking apparatus comprising: at least one processor; a memory storing instructions executable by the at least one processor to ([0038], [0060], [0068]); and non-transitory computer readable medium storing a program executable by a computer to perform an object tracking method ([0038], [0279]) comprising: receiving a request for specifying a first object (As with vehicles, the machine vision associated with individuals may be enhanced during the period of system configuration, for example by having a resident stand in front of the camera, so that the system can learn to recognize a resident on the basis of physical identification characteristics, [0096], It should be understood that these methods of arming and disarming the system, or increasing or decreasing the level of risk associated with a particular set of sensor conditions may be user-configurable, may be configured by remote personnel, by a local installer, or by default settings. In particular, the presence of a phone on the premises may normally be considered to be evidence of the presence of a homeowner, but a homeowner who leaves a phone at home accidentally when heading to work may remotely signal the occurrence of this event (for example through an online interface, or by calling in a particular code), which may then alert the system to ignore the presence of the phone for a period of time, [0132], In combination with other forms of recognition of individuals, gesture-based signals may be provided to the vision system to disarm the system, Alternatively, or additionally, a set of physical characteristics of an individual, such as height, extremity lengths, skin color, or facial features may be utilized as a means of identifying individuals. When the security system observes a specific set of attributes and/or behaviors, the security system may be configured to disarm, as described above, [0170], In the event that a potential entry by an unidentified person is observed, a primary contact may be notified, [0175], David Smith has requested to be notified if his car is observed moving through a location without his cell phone present, [0233], The camera monitoring systems may include visual displays. For residents returning at night, the displays might be tailored to say "Welcome back, John", [0261]) [first object interpreted as each respective homeowner]; determining the first object captured in a video as an object to be tracked (extracting identifying features of passengers in vehicles, such as the number of passengers, the orientation of a driver or passenger's eyes, the size or gender of passengers, facial images may also be stored or analyzed for the purpose of facial recognition and positive identification, [0082], Machine vision may also allow the system to identify people, to measure physical dimensions of individuals, to observe the direction of eye movements, or to recognize faces or gestures, as examples, [0096]); associating the first object with a second object having a predetermined relationship with the first object (The footage may be utilized by a user to identify the vehicle, a person associated with a vehicle, or a risk level. For example, a homeowner may identify a vehicle as "Aunt Jane's Car" or a monitoring company may identify a vehicle as a "suspicious vehicle". Features associated with vehicles and identified by users may be stored in a database, and accessed in future visits by a vehicle in predicting the vehicle's location, [0066], Mrs. Jones and her husband were observed in their cars exiting the neighborhood earlier that day, and have not been observed re-entering the neighborhood at any point since, [0135]); and tracking the first object in a case where the first object and the second object are detected in the video (From David Smith's home security system: a video feed showed a white Dodge Dakota with license XYZ 123 approaching the home at 12:15 PM; a video feed showed a passenger leaving the car and entering the home at 12:16 PM; an entry to the home was noted by a door proximity switch at 12:16 PM; the appropriate security code was entered on a home security system at 12:17 PM, [0229], David Smith's cell phone should travel into and out of the neighborhood with his car. When observed to be consistent with expected pattern, the situation is evaluated as normal, [0236], Obviously, in the case above, we selected an instance where the situation observed was of a neighborhood resident and followed an expected pattern, [0241]).
Strimling does not explicitly disclose receiving a request for specifying a first object.
Manasseh et al. teach receiving a request for specifying a first object (Agents may use computers to enter diverse relevant information about each traveler, [0022], responding to an alarm in pre-defined manner, communicating with the various service and control stations, communicating with external resources (law enforcement agencies), setting predetermined rules and actions for the control system to follow, [0034], 9) Searching device 236 for enabling an operator at the control room to search the database for a specific traveler, follow the traveler's itinerary in the airfield or other locations, and correlate certain data items to find discrepancies, [0036], . A specific security application use the data stored in database to analyze, for example, the following data, having relevance on issues of flight security: a) Cross check the given identity and appearance of the traveler with databases of, for example, the Interpol or FBI related to "wanted" persons, [0047]); determining the first object captured in a video as an object to be tracked (video cameras 142, 144 that capture a sequentially ordered sequence of images visually reflecting the agent-traveler interaction, captured video images are sent via the communications line 101 to the control room 150 or to a remote location in order to be processed, to be integrated with the traveler 146 record, to be recorded, stored and optionally to be retrieved, [0028], The traveler 118 may be monitored when he boards or disembarks a vehicle such as an airplane via the capture devices located on board such vehicle, [0029]); associating the first object with a second object having a predetermined relationship with the first object (In a transportation vehicle such stations could include a boarding point, a passenger cabin, and the like. The major interaction capturing locations 12, 14, 16, 18, 20 are linked through a communications network 22 to a recording and retrieval system 24, [0024]); and tracking the first object in a case where the first object and the second object are detected in the video (agents may provide positive identification of passengers at certain points along the path of the passenger, It may be extended to hotels, vehicles of transportations (airplanes, ships, and the like), and other locations where traveler-agent interactions exist and may be captured, [0022], Likewise the traveler may be monitored during boarding until he reaches his allotted seat, thus eliminating the need to manually account for his location on the vehicle, [0029], successive monitoring of the agent-traveler-baggage interaction, [0032], a traveler checks-in to a flight having a particular handbag and later at the boarding he or she carries a different handbag, [0036], The system stores data on the same traveler going through the process of boarding the airplane. Data is collected and updated once the traveler is processed at a specific station, b) Verify (tag and visual) that the same traveler passed all designated stations, [0047]).
Strimling and Manasseh et al. are in the same art of object tracking (Strimling, [0229], [0236];
Manasseh et al., [0022], [0047]). The combination of Manasseh et al. with Strimling will enable using a request for specifying. It would have been obvious at the time of filing to one of ordinary skill in the art to combine the request of Manasseh et al. with the invention of Strimling as this was known at the time of filing, the combination would have predictable results, and as Manasseh et al. indicate this will improve travel safety ([0008]-[0010]) suggesting a safety benefit to combining the inventions.
Regarding claim 2, Strimling and Manasseh et al. disclose the object tracking method according to claim 1. Strimling and Manasseh et al. further indicate the predetermined relationship includes that the first object enters into the second object or the first object rides on the second object (Strimling, extracting identifying features of passengers in vehicles, such as the number of passengers, the orientation of a driver or passenger's eyes, the size or gender of passengers, facial images may also be stored or analyzed for the purpose of facial recognition and positive identification, [0082], Mrs. Jones and her husband were observed in their cars exiting the neighborhood earlier that day, and have not been observed re-entering the neighborhood at any point since, [0135], From David Smith's home security system: a video feed showed a white Dodge Dakota with license XYZ 123 approaching the home at 12:15 PM; a video feed showed a passenger leaving the car and entering the home at 12:16 PM; an entry to the home was noted by a door proximity switch at 12:16 PM; the appropriate security code was entered on a home security system at 12:17 PM, [0229]; Manasseh et al., Likewise the traveler may be monitored during boarding until he reaches his allotted seat, thus eliminating the need to manually account for his location on the vehicle, [0029], The system stores data on the same traveler going through the process of boarding the airplane, [0047]).
Regarding claim 6, Strimling and Manasseh et al. disclose the object tracking method according to claim 1. Strimling and Manasseh et al. further indicate the first object is a person (Strimling, the machine vision associated with individuals may be enhanced during the period of system configuration, for example by having a resident stand in front of the camera, so that the system can learn to recognize a resident on the basis of physical identification characteristics, [0096], a set of physical characteristics of an individual, such as height, extremity lengths, skin color, or facial features may be utilized as a means of identifying individuals, [0170], In the event that a potential entry by an unidentified person is observed, a primary contact may be notified, [0175], David Smith has requested to be notified if his car is observed moving through a location without his cell phone present, [0233], The camera monitoring systems may include visual displays. For residents returning at night, the displays might be tailored to say "Welcome back, John", [0261]; Manasseh et al., Agents may use computers to enter diverse relevant information about each traveler, [0022], Likewise the traveler may be monitored during boarding until he reaches his allotted seat, thus eliminating the need to manually account for his location on the vehicle, [0029]).
Regarding claim 7, Strimling and Manasseh et al. disclose the object tracking method according to claim 1. Manasseh et al. further indicate the first object is a baggage (Video and other cameras may also capture the appearance of the passenger as well as other important features, such as his texture, cloths, demeanor, accompanying persons, size, and appearance of baggage and other information, which may be captured by cameras, links between baggage and one or more travelers or persons may also be easily established [0022], The sequence of captured images reflects the visual characteristics of the baggage 180, [0027] analysis of the video images or the audio record captured to see whether the traveler changed his appearance or collected different articles, such as a new handbag, or a an additional baggage and the like, [0031]).
Regarding claim 8, Strimling and Manasseh et al. disclose the object tracking method according to claim 1. Strimling and Manasseh et al. further indicate the second object is a vehicle (Strimling, extracting identifying features of passengers in vehicles, [0082], Mrs. Jones and her husband were observed in their cars exiting the neighborhood earlier that day, and have not been observed re-entering the neighborhood at any point since, [0135], From David Smith's home security system: a video feed showed a white Dodge Dakota with license XYZ 123 approaching the home at 12:15 PM; a video feed showed a passenger leaving the car and entering the home at 12:16 PM; an entry to the home was noted by a door proximity switch at 12:16 PM; the appropriate security code was entered on a home security system at 12:17 PM, [0229]; Manasseh et al., Likewise the traveler may be monitored during boarding until he reaches his allotted seat, thus eliminating the need to manually account for his location on the vehicle, [0029], The system stores data on the same traveler going through the process of boarding the airplane, [0047]).
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Strimling (US 20140334684 A1) and Manasseh et al. (US 20050128304 A1) as applied to claim 1 above, further in view of Senior (US 20110317920 A1).
Regarding claim 3, Strimling and Manasseh et al. disclose the object tracking method according to claim 1. Strimling and Manasseh et al. do not disclose the predetermined relationship includes that the first object and the second object overlap each other and then the first object is not detected from the video data without being separated from the second object.
Senior teaches the predetermined relationship includes that the first object and the second object overlap each other and then the first object is not detected from the video data without being separated from the second object (Occlusion determinator 48 determines whether two objects may be occluding one another. In making this determination, occlusion determinator 48 may, based on the positions of the two objects within video image 19, and the detected foreground region or regions near to the objects, conclude that the objects are sufficiently close that a likelihood exists that one object occludes the other. Occlusion determinator 48 may also compare current attributes of two or more objects in the series of video images and their object models. In doing so, occlusion determinator 48 may make use of the object model that has been previously constructed by object model constructor 44 to make a determination that an occlusion has occurred. Still further, if the expected paths based on the temporal association created by object tracker 46 of two or more objects indicate that the objects would overlap or nearly overlap in the new frame, then occlusion determinator 48 may conclude that an occlusion has occurred, [0037], Occlusion determinator 48 determines whether two objects may be occluding one another. In making this determination, occlusion determinator 48 may, based on the positions of the two objects within video image 19, and the detected foreground region or regions near to the objects, conclude that the objects are sufficiently close that a likelihood exists that one object occludes the other. Occlusion determinator 48 may also compare current attributes of two or more objects in the series of video images and their object models. In doing so, occlusion determinator 48 may make use of the object model that has been previously constructed by object model constructor 44 to make a determination that an occlusion has occurred. Still further, if the expected paths based on the temporal association created by object tracker 46 of two or more objects indicate that the objects would overlap or nearly overlap in the new frame, then occlusion determinator 48 may conclude that an occlusion has occurred, [0039], Referring again to FIGS. 1, 2 and 3 concurrently, region analyzer 50 performs an analysis of an unresolved region to associate the unresolved region with one of the objects or to assign the region as ambiguous, [0040]) [assigning interpreted as detecting, occlusion interpreted as overlapping].
Strimling and Manasseh et al. and Senior are in the same art of object tracking (Strimling, [0229], [0236]; Manasseh et al., [0022], [0047]; Senior, [0009]). The combination of Senior with Strimling and Manasseh et al. will enable defining the predetermined relationship includes that the first object and the second object overlap each other and then the first object is not detected from the video data without being separated from the second object. It would have been obvious at the time of filing to one of ordinary skill in the art to combine the relationship of Senior with the invention of Strimling and Manasseh et al. as this was known at the time of filing, the combination would have predictable results, and as Senior indicate the invention will resolve occlusions in vehicle tracking ([0006], [0017]) thereby the combination of inventions should make the tracking process of Strimling and Manasseh et al. more accurate.
Claim(s) 4-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Strimling (US 20140334684 A1) and Manasseh et al. (US 20050128304 A1) as applied to claim 1 above, further in view of Tserng (US 6570608 B1).
Regarding claim 4, Strimling and Manasseh et al. disclose the object tracking method according to claim 1. Strimling and Manasseh et al. do not disclose tracking the second object in a case where, after associating the first object with the second object, the first object is not detected but the second object is detected.
Tserng teaches tracking the second object in a case where, after associating the first object with the second object, the first object is not detected but the second object is detected (“Note that this current car image includes an occluding portion of a person. FIG. 6C illustrates the car difference image, including the car and the occluding portion of the person… The blobs in the car difference image are stored. Note that every blob in the car difference image necessarily represents the car because this includes only the subimages are the location of the car”, col. 10, lines 1-15, “FIGS. 13A to 13E illustrate an example of this technique. In FIGS. 13A to 13E the person in the previous example finally enters the car. FIG. 13A illustrates the current video image. FIG. 13B illustrates the foreground difference image. This shows a person blob due to the fake background in the updated reference image not matching the person who has just entered the car. FIG. 13C illustrates the reference car image with the fake background. Because this blob in FIG. 13B includes more that 70% black pixels in the car reference image in FIG. 13C, this reference car image is corrected. FIG. 13D illustrates the current car image, which is used to correct the reference car image. FIG. 13E illustrates the corrected reference car image”, col. 12, lines 20-35, “The second aspect of the invention that we wish to protect is the specific technique used to track people and vehicles during interactions… When the car comes to rest, its image is temporarily inserted into the reference image. This allows the system to track people when they move in front of or behind the car, and to detect changes in the car shape due to doors opening, etc. When the car begins to move again, the system restores the reference image to its original state and tracks the car using the older tracking method”, col. 16, lines 25-30).
Strimling and Manasseh et al. and Tserng are in the same art of object tracking (Strimling, [0229], [0236]; Manasseh et al., [0022], [0047]; Tserng, abstract). The combination of Tserng with Strimling and Manasseh et al. will enable tracking the second object in a case where, after associating the first object with the second object, the first object is not detected but the second object is detected. It would have been obvious at the time of filing to one of ordinary skill in the art to combine the tracking of Tserng with the invention of Strimling and Manasseh et al. as this was known at the time of filing, the combination would have predictable results, and as Tserng indicates this will reduce need for manual labeling and work in real time to detect suspicious events (col. 1, lines 25 – 50) thereby improving the safety of the combination of inventions.
Regarding claim 5, Strimling and Manasseh et al. and Tserng disclose the object tracking method according to claim 4. Tserng further indicate tracking the first object in a case where the first object is detected after tracking the second object (An EXIT CAR event is signaled whenever an object overlaps a stationary car in one image and the corresponding object is not present in the immediately prior image, col. 2, lines 35-40, track each object through the video sequence, col. 6, lines 60-65, “In FIG. 3, the nineteen vertical lines F0 through F18 each represent a respective frame or image in a series of successive images from the video camera 12. In FIG. 3, the horizontal dimension represents time, and the vertical dimension represents one dimension of movement of an object within a two-dimensional image. When an object which was not previously present first appears, for example at 51 or 52, it is identified as an entrance or ENTER event. When an object which was previously present is found to no longer be present, for example at 53 or 54, it is designated an EXIT event. If an existing object splits into two objects, one of which is moving and the other of which is stationary, for example as at 57, it is designated a DEPOSIT event. This would occur, for example, when a person who is carrying a briefcase sets it down on a table, and then walks away”, col. 7, lines 45-50, see Fig. 3, Using this technique it is easy to detect when people enter and exit a car. If an object disappears when its overlapped with a car, it probably entered the car. Similarly, if an object appears overlapped with a car, it probably exited the car, col. 9, lines 25-35, The next Figures illustrate an example of a person exiting a car, which then moves away. FIG. 14A illustrates the foreground difference image in which a person has just exited a resting car. As seen in the foreground difference image of FIG. 14A, the system recognizes the person as a real object. FIG. 14B illustrates the corresponding final output image. Finally, when the car moves away and the person blob is separated from the car blob. The system compares the new person blob with the blob which was stored earlier. The new blob is similar in appearance and at the same location as the stored blob. Then the new blob is linked to this object and object integrity is maintained, col. 12, line 65 – col. 13, line 25, FIGS. 18A to 18D illustrates an example of a person exiting a car, col. 14, lines 1-5).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE ENTEZARI whose telephone number is (571)270-5084. The examiner can normally be reached 10-7 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent M Rudolph can be reached at (571) 272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE M ENTEZARI HAUSMANN/Primary Examiner, Art Unit 2671
Prosecution Insights