DETAILED ACTION
Notice of Pre-AIA or AIA Status
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 02/13/2026. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
AMENDMENTS
Applicant/s submitted arguments and remarks on 02/13/2026. The Examiner acknowledges the arguments and reviewed the claims accordingly.
Applicant/s amended claims 1, 7 and 8. Claims 2, 3, 6 and 16 have been cancelled. Claims 1, 4 – 5, and 7 – 15 are currently pending.
Response to Arguments
In regards to Argument 1, with respect to the rejection of claim 1 – 16 under 35 U.S.C. 101 for being directed to an abstract idea, the applicant/s states that independent claims 1 and 8 has been amended to overcome the rejection. The applicant/s further states that the amended independent claim 8 recited additional limitations that amount to significantly more than the judicial exception. Therefore, the applicant/s requests withdrawal of objection of claims. (See Arguments/Remarks, page 7 – 10, dated 02/13/2026)
In response to Argument 1, with respect to the rejection of claim 1 – 16 under 35 U.S.C. 101 for being directed to an abstract idea, the Examiner states that the applicant/s arguments have been fully considered but are rendered moot in view of the amendments made to the independent claims 1 and 8. However, upon further consideration, amended independent claim 8 is found to be reciting significantly more than an abstract idea by tracking movement of the identified objects across images with increased confidence in response to detecting that the identified object moves across the plurality of the images from first camera and second cameras in the same direction using the stored time stamp and bounding box by correlating directions of the identified object in the images using the detection history. Therefore, the Examiner states that rejection of independent claim 8 and their dependent claims 9 – 15 under 35 U.S.C. 101 for abstract idea have been withdrawn. However, upon further consideration, the Examiner states that the amended independent claim 1 and dependent claims 4, 5 and 7 do not recite additional elements that amount to significantly more than an abstract idea and therefore are not patent eligible under 35 U.S.C. 101. The Examiner further states that the use of an object detection model that is trained using machine learning is recited in the amended claim 1 at a high level of generality and the claim does not recite the method or steps of training and it further does not reflect any improvement in the functioning of a computer, or an improvement to other technology or technical field. An object detection model, under broadest reasonable interpretation, could be a generic computer program or components configured to perform the method. Therefore, the examiner is making the following new grounds of rejection for claims 1, 4 – 5 and 7 under 35 U.S.C. 101 as described below.
In regards to Argument 2, with respect to the rejection of claim 8 under 35 U.S.C. 102, the applicant/s states that independent claim 8 has been amended. The applicant/s further states that Biancale fails to teach the limitations recited in the amended independent claim 8. Therefore, the applicant/s requests withdrawal of rejection of claim 8 under 35 U.S.C. 102. (See Arguments/Remarks, page 10 – 11, dated 02/13/2026)
In response to Argument 2, with respect to the rejection of claim 8 under 35 U.S.C. 102, the Examiner states that the applicant/s arguments have been fully considered but are rendered moot in view of the amendments made to the independent claim 8. Therefore, the examiner states that the rejection of claim 8 under 35 U.S.C. 102 has been withdrawn. However, upon further search and consideration, the following new grounds of rejections have been necessitated by the amendments.
In regards to Argument 3, with respect to the rejection of claims 1 – 2, 5, 6, 9 and 12 – 14, under 35 U.S.C. 103, the applicant/s states that independent claim 1 has been amended and claims 2, 3 and 6 have been cancelled. The applicant/s further states that Biancale and Flick fail to teach the limitation recited in the amended independent claim 1. Therefore, the applicant/s requests withdrawal of rejection of claims under 35 U.S.C. 103. (See Arguments/Remarks, page 11 – 13, dated 02/13/2026)
In response to Argument 3, with respect to rejection of claims 1 – 2, 5, 6, 9 and 12 – 14, under 35 U.S.C. 103, the Examiner states that the applicant/s arguments have been fully considered but are rendered moot in view of the amendments made to the independent claim 1. Therefore, the Examiner states that rejection of claims under 35 U.S.C. 103 have been withdrawn. However, upon further search and consideration, independent claim 1 and its dependent claims 4, 5 and 7 have been found to be allowable over prior art.
Claim Objections
Claim 1 and 8 are objected to because of the following informalities:
Claim 1, firth limitation recites “the model”. It should recite as “the object detection model”.
Claim 8, replace period “.” at the end of tenth limitation with a semicolon “;”: “…detection history for the identified object[[.]];”
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4, 5 and 7 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process (concept performed in a human mind, including as observation, evaluation, judgment, opinion, organizing human activity and mathematical concepts and calculations). The claim(s) recite(s) a method, detect a focus of attention. This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved .The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally and no additional features in the claims would preclude them from being performed as such except for the generic computer elements at high level of generality (i.e., processor, memory).
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claims 1 and 10 are directed to an abstract idea as shown below:
STEP 1: Do the claims fall within one of the four statutory categories? YES. Claim 1 is directed to a system.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? YES, the claims recite steps that fall into the abstract idea category of mental processes.
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
The system in claim 1 comprise a mental process that can be practicably performed in the human mind (or generic computers or components configured to perform the method) and, therefore, an abstract idea.
Regarding Claim 1: A system comprising: a processor; and a memory storing instructions which when executed by the processor configure the processor to:
receive a plurality of images captured by a camera from an image processing system (insignificant extra solution activity of data gathering);
detect an object in an image from the plurality of images using an object detection model that is trained using machine learning (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; object detection model that is trained using machine learning can be any generic computer program);
identify the detected object using the model and a database of previously identified objects (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; identify the detected object…);
track movement of the identified object across a series of images from the plurality of images (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; track movement of the identified object across a series of images…); and
detect, based on the plurality of images, when the identified object disappears from view of the camera (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; detect identified objects…); and
determine an outcome for the identified object based on first and last detections of the identified object and a direction of movement of the identified object (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; deriving…);
detecting the identified object in N1 images from the plurality of images, where N1 is an integer greater than 1 (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; detecting objects in images…);
determining, when the identified object disappears after the N1 images but reappears in less than or equal to N2 images of the plurality of images following the N1 images, that the identified object detected in the N2 images is a continued detection of the identified object detected in the N1 images (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; detecting objects in images…for example id N1 is 2 images and N2 is 3 images following N1, it is possible to identify the object detected in N2 images is a continued detection of the identified object detected in the N1 images by observation and evaluation and hence a mental process);
when the identified object is detected in N1 + N2 images, determining that the identified object remains in an area under observation, and determining that the identified object has left the area under observation when the identified object is not detected in N1 + N2 images (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; detecting objects in images…for example id N1 is 2 images and N2 is 3 images following N1, it is possible to mentally identify if object is detected in 2 + 3 (= 5) images by a human and hence a mental process), and
wherein, for each instance of detection of the identified object, the instructions further configure the processor to:
assign a timestamp to the detection of the identified object (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; assigning time stamp…);
assign a label to the identified object (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; assigning a label…);
assign bounding box coordinates for the identified object ((mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; assigning bounding box… can be done using a generic computer component or a paper and a pen); and
store the timestamp, the label, and the bounding box coordinates in a detection history for the identified object (mental process including observation and evaluation, and can be done mentally in the human mind or a generic computer program or components configured to perform the method; storing…).
These limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that “can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same). As such, a person could mentally analyze an image and determine a fill level, either mentally or using a pen and paper. The mere nominal recitation that the various steps are being executed by a device/in a device (e.g. processing unit) does not take the limitations out of the mental process grouping.
The claimed functions –receiving data, detecting, identifying, tracking and determining– could be performed conceptually by a human using pen and paper or using a generic computer component, and thus fall under abstract mental steps.
Conclusions: Thus, the claims are directed to an abstract idea.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? NO, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim 1 does/do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
These limitations are recited at a high level of generality (i.e. as a general action or change being taken based on the results of the acquiring step) and amounts to mere post solution actions, which is a form of insignificant extra-solution activity. Further, the claims are claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception.
There is no indication that the method improves the functioning of a computer or classification itself.
Conclusion: Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? NO, the claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Claim 1 does/do not recite any additional elements that are not well-understood, routine or conventional.
The claims lack an inventive concept sufficient to transform the abstract idea into patent-eligible subject matter.
The claims are functionally generic with no details about architecture, dataset specifics, or a novel arrangement of components.
Conclusion: The claims does not add significantly more than the abstract idea.
Final Determination: INELIGIBLE under 35 U.S.C. 101. The Claim 1 is: directed toward an abstract idea (mental process and data manipulation) using conventional tool (model) in a generic way, without integration into a practical application or an inventive concept.
Regarding Claims 4, 5 and 7: the additional elements recited in the claims do not integrate the mental process into a practical application or add significantly more to the mental process. The limitations merely recite that the functions are performed by a processing unit and does not demonstrate a technological improvement. The claims are functionally generic with no details about architecture, dataset specifics, or a novel arrangement of components. Since the claims are directed toward an abstract idea (mental process and data manipulation) using conventional tools in a generic way, without integration into a practical application or an inventive concept, they are ineligible under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 8 – 15 are rejected under 35 U.S.C. 103 as being unpatentable over Glaser et al. (US 20190378205 A1; hereafter referred to as Glaser) in view of Olgiati (US 20190050629 A1; hereafter referred to as Olgiati).
Regarding Claim 8, Glaser teaches:
A system comprising:
a processor (Glaser, [0265] “The computer-executable component can be a processor”); and
a memory storing instructions which when executed by the processor configure the processor (Glaser, [0265] “he computer-readable medium can be stored on any suitable computer readable media such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD or DVD), hard drives, floppy drives, or any suitable device”) to:
receive images captured by first and second cameras from an image processing system (Glaser, [0070] “The imaging system 100 preferably includes a set of image capture devices 110 (e.g., digital cameras). An image capture device 100 might collect some combination of visual, infrared, depth-based, lidar, radar, sonar, and/or other types of imagery”);
detect an object in an image from the plurality of images using an object detection model that is trained using machine learning (Glaser, [0058] The system and method preferably applies an environmental object graph (EOG) modeling system, which can be used to probabilistically detect, track, and model objects in space and time across an environment”; Glaser, [0085] “Classification can leverage use of image feature extraction and classification, statistical machine learning, neural networks”);
identify the detected object using the model and a database of previously identified objects (Glaser, [0091] The system can additionally include a reference object database that functions to provide object property information. Object property information may in some variations be used in modeling objects through the EOG system 200”);
for each instance of detected and identified object, the instructions configure the processor to:
assign a timestamp to the detection of the identified object (Glaser, [0139] “the modeled object instances in an EOG can have object classifications, location, timestamps, and/or other properties such as SKU or product associated properties”);
assign a label to the identified object (Glaser, [0147] “Resulting output of classifying objects of image data of a single image or video stream can be a label or probabilistic distribution of potential labels of objects, and a region/location property of that object”);
However, Glaser does not explicitly teach:
assign bounding box coordinates for the identified object;
store the timestamp, the label, and the bounding box coordinates in a detection history for the identified object;
detect the identified object with increased confidence in one of the images from the first camera in response to detecting the identified object moving across a plurality of the images from the first and second cameras in a same direction; and
track movement of the identified object across the images with increased confidence in response to detecting that the identified object moves across the plurality of the images from the first camera and second cameras in the same direction using the stored timestamp and bounding box and by correlating detections of the identified object in the images using the detection history.
In the same field of endeavor, Olgiati teaches:
assign bounding box coordinates for the identified object (Olgiati, [0011] “the object detector is a software component that includes executable code that is executable to receive and ingests multimedia content to generate object tracking metadata such as bounding boxes for individuals moving through a scene”; Olgiati, [0016] “a system such as a multi-trait identifier receives object tracking metadata which includes data regarding the location of the tracked object (e.g., data such as coordinates for a bounding box around the object), the direction and velocity of the tracked object”); and
store the timestamp, the label, and the bounding box coordinates in a detection history for the identified object (Olgiati, [0014] “the identity association metadata is embedded directly into the multimedia content (e.g., in a data file or data structure that was created with the multimedia content), encoded in existing metadata associated with the multimedia content (e.g., stored as an extension to an existing manifest file), stored separately from and in association with the multimedia content (e.g., as a metadata that can be accessed separately from the multimedia content)”);
detect the identified object with increased confidence in one of the images from the first camera in response to detecting the identified object moving across a plurality of the images from the first and second cameras in a same direction (Olgiati, [0016] “a process for selectively performing an identity recognition process based on object tracking metadata is implemented …for example, is implemented by a multi-trait identifier. … a system such as a multi-trait identifier receives object tracking metadata which includes data regarding the location of the tracked object (e.g., data such as coordinates for a bounding box around the object), the direction and velocity of the tracked object, whether the object is partially or wholly occluded, and other metadata that is usable to determine a confidence score of an identity associated to the object”); and
track movement of the identified object across the images with increased confidence in response to detecting that the identified object moves across the plurality of the images from the first camera and second cameras in the same direction using the stored timestamp and bounding box and by correlating detections of the identified object in the images using the detection history (Olgiati, [0031] “ receive object metadata for multiple tracked objects being tracked in a video where object metadata for a tracked object includes information regarding the tracked object which further includes, for example, information regarding the direction the object is moving, the orientation of the object, and a bounding box such as those described elsewhere in connection with FIG. 4 that is used to track the object as it moves throughout the frame. .. the multi-trait identifier 202 requests object metadata from an object detector …and the object detector generates object metadata 204 that includes coordinates for a bounding box around an object, the direction the object is moving, the orientation of the object (e.g., facing towards the video, facing away from the video, lateral to the video), whether the object is occluded, whether the object overlaps with another object (e.g., based on the bounding boxes tracking the objects), and more”; Olgiati, [0037] “the diagram shows a first frame 304 of a video at a first point in time (as denoted by the timestamp 310 of the video shown in the lower right hand corner of the first video frame 1:11, and subsequent frames at later times), a second frame 306 of the video at a second point in time (i.e., at 1:22 in the video, eleven seconds after the first frame 304”).
Glaser and Olgiati are considered analogous art as they are reasonably pertinent to the same field of endeavor of tracking objects. Therefore, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Glaser with the method of detecting the identified object as taught by Olgiati to make the invention that assigns bounding box coordinates for the identified object; stores the timestamp, the label, and the bounding box coordinates in a detection history for the identified object; detects the identified object with increased confidence in one of the images; and tracks movement of the identified object across the images with increased confidence in response to detecting that the identified object moves across the plurality of the images; doing so can efficiently recognize and identify moving objects and track the movement of the object (Olgiati [0001]); thus, one of the ordinary skill in the art would have been motivated to combine the references.
Regarding Claim 9, Glaser in view of Olgiati teaches the system of claim 8 wherein the instructions configure the processor to detect the identified object with increased confidence in one of the images from the first camera in response to detecting the identified object in one of the images from the second camera (Olgiati, [0038] “ association data between a tracked object and identity information is generated and maintained which includes, for example, metadata such as a confidence score corresponding to the strength of the association, an association state that indicates whether an association exists, and other data that can be utilized to evaluate whether an object is substantially identifiable as well as evaluate whether a recognition process should be performed by monitoring object tracking metadata, determining a confidence score or an association state, determining that the confidence score or the association state changed based on received metadata that causes the confidence score to fall below a threshold score or an association state to change to a state which indicates an identity recognition process should be performed”).
Regarding Claim 10, Glaser in view of Olgiati teaches the system of claim 8 wherein the instructions configure the processor to:
detect that the identified object moves across a plurality of the images from the first and second cameras in the same direction (Glaser, [0074] “The interaction capture configuration may include a visual video camera that is directed in a direction substantially coplanar with the face of a shelf so as to show how a person interacts with an inventory storage compound. For example, in a store, a camera in an interaction capture configuration may be directed down an aisle. A series of imaging devices can be positioned periodically down an aisle, wherein each image capture device is intended for detecting interaction events with subsections of the aisle”; Glaser, [0076] “one or more imaging systems 10o may be mounted to a robot or person as shown in FIG. 8, and collect image data as the imaging device is moved through the environment); and
track the movement of the identified object with increased confidence in response to detecting that the identified object moves across the plurality of the images from the first and second cameras in the same direction (Glaser, [0055] “the system and method can be used in tracking items within a store such that the location of an item can be queried”; Glaser, [0069] “The system primarily functions to track and account for physical items within an environment”; Glaser, [0143] “an object identification imaging device may be actuated so as to track or target particular objects. Object identification capture configurations may additionally or alternatively be used in a checkout or POS configuration variation. A checkout variation functions to collect object information for identification during a checkout process. A camera in a checkout configuration may be particularly configured to capture items as they are put on a conveyor belt or set out for inspection by a worker”).
Regarding Claim 11, Glaser in view of Olgiati teaches the system of claim 10 wherein the instructions configure the processor to predict subsequent detections of the object in the direction of movement with increased confidence (Glaser, [0062] “the EOG system preferably enables predicting or modeling probabilities of contained objects. When combined with interaction event detection this can be used in accounting for a perceived appearance of an object, disappearance of an object, splitting of an object, or merging of object”; Glaser, [0063] “The probabilistic modeling can be updated through updated information to more confidently identify most likely state, and/or resolve the modeling into a predicted state when requested for application operations”).
Regarding Claim 12, Glaser in view of Olgiati teaches the system of claim 8 wherein the instructions configure the processor to:
detect when the identified object disappears from view of the first camera (Glaser, [0042] “a small shelf of items could be monitored such that item selection by a shopper can be tracked and then an automatic checkout process can execute when the shopper leaves the field of view or proceeds through an exit as shown in FIG. 4”); and
track the movement of the identified object in a plurality of the images from the second camera in response to the identified object disappearing from view of the first camera (Glaser, [0170] “tracking objects preferably includes constructing an object path, where an object path establishes a continuous path of an object in time and space. An object path can include stationary paths or path segments to account for an object being stationary like a product on a shelf. At some instance, continuous observation of an object may be prevented and an object path will terminate. Terminal points of an object path preferably include a point of appearance and a point of disappearance”).
Regarding Claim 13, Glaser in view of Olgiati teaches the system of claim 12 wherein the instructions configure the processor to:
detect when the identified object disappears from view of the second camera (Olgiati, [0031] “the object detector generates object metadata 204 that includes coordinates for a bounding box around an object, the direction the object is moving, the orientation of the object (e.g., facing towards the video, facing away from the video, lateral to the video), whether the object is occluded, whether the object overlaps with another object (e.g., based on the bounding boxes tracking the objects), and more”; Glaser, [0187] “Occlusion of object A may trigger an interaction event that updates an object A as possibly being in the shelf or in selected by the shopper. If the shopper walks away, that interaction event in combination with the disappearance of the earlier object A increases the probability that object A was selected by the shopper”);
determine first and last detections of the identified object and a direction of movement of the identified object in the images from the first and second cameras (Glaser, [0074] “The interaction capture configuration may include a visual video camera that is directed in a direction substantially coplanar with the face of a shelf so as to show how a person interacts with an inventory storage compound. For example, in a store, a camera in an interaction capture configuration may be directed down an aisle. A series of imaging devices can be positioned periodically down an aisle, wherein each image capture device is intended for detecting interaction events with subsections of the aisle”); and
determine an outcome for the identified object based on the first and last detections of the identified object and the direction of movement of the identified object (Glaser, [0182] The result of an interaction event may be known wherein the nature of the interaction is visually observed…. For example, a shopper reaching into a shelf and interacting with an item may be considered as a transfer of an object from the shelf to the shopper with a first probability, the shopper transferring an object to the shelf with a second probability and the shopper removing a different object with a third probability. Sometimes the detection of an interaction event is not coincident with its understanding. In these cases, a subsequent inspection of the shelf and shopper may offer the best information about what objects were transferred and in which direction”).
Regarding Claim 14, Glaser in view of Olgiati teaches the system of claim 13 wherein determining the outcome includes determining that the identified object remains in an area being observed or that the identified object has moved out the area being observed (Glaser, [0088] “The EOG processing engine 210 can additionally include an interaction process, which functions to draw conclusions on the state of items in an environment based on object interactions. An interaction process can involve two or more objects, which generally is associated with proximity, contact, and/or changes in objects in some region between objects. For example, the representation of two objects can be updated when the objects make contact or are moved to be within a particular proximity… the path of a shopper through the store can alter the EOG of the shopper object based on proximity to particular objects in the store…. Object transformations generally show the transition of a first set of objects into a second set of objects… As shown in FIG. 10, examples of basic object transformations related to interaction events may include revealing contents of an object, concealing an object by another object, splitting an object into more than one object, merging a set of objects together, appearance of an object, and disappearance of an object”).
Regarding Claim 15, Glaser in view of Olgiati teaches the system of claim 13, wherein the instructions configure the processor to:
for each instance of detection of the identified object:
assign a timestamp to the detection of the identified object (Olgiati, [0037] “The diagram shows a first frame 304 of a video at a first point in time (as denoted by the timestamp 310 of the video shown in the lower right hand corner of the first video frame 1:11, and subsequent frames at later times), a second frame 306 of the video at a second point in time (i.e., at 1:22 in the video, eleven seconds after the first frame 304), a third frame 308A of the video at a third point in time, and the third frame 308B of the video after a multi-trait identifier 302 performs one or more processes relating to associating objects of the video to identities”);
assign a label to the identified object (Glaser, [0147] “Resulting output of classifying objects of image data of a single image or video stream can be a label or probabilistic distribution of potential labels of objects, and a region/location property of that object”);
assign bounding box coordinates for the identified object (Olgiati, [0011] “the object detector is a software component that includes executable code that is executable to receive and ingests multimedia content to generate object tracking metadata such as bounding boxes for individuals moving through a scene”; Olgiati, [0016] “ a system such as a multi-trait identifier receives object tracking metadata which includes data regarding the location of the tracked object (e.g., data such as coordinates for a bounding box around the object), the direction and velocity of the tracked object”); and
store the timestamp, the label, and the bounding box coordinates in a detection history for the identified object (Olgiati, [0014] “the identity association metadata is embedded directly into the multimedia content (e.g., in a data file or data structure that was created with the multimedia content), encoded in existing metadata associated with the multimedia content (e.g., stored as an extension to an existing manifest file), stored separately from and in association with the multimedia content (e.g., as a metadata that can be accessed separately from the multimedia content)”); and
determine, by correlating the detection history for the identified object, the first and last detections of the identified object and the direction of movement of the identified object (Glaser, [0089] “The EOG processing engine 210 may additionally have an associative propagation process, which functions to draw conclusions on the state of items in the environment based on observations over a history of observations. The propagation process generally relies on input collected over a longer period of time in updating the EOG than the interaction or static processes. In a propagation process, updates to one part of the EOG can propagate through historical object associations and relationships to update a second part of the EOG at one or more other instances in time or space. The propagation process is preferably used in combination with the static process and the interaction process”).
Allowable Subject Matter
Claims 1, 4, 5 and 7 are objected to but would be allowable by overcoming the rejection of claims 1, 4, 5 and 7 under 35 U.S.C. 101 and overcoming any claims objections.
Claims 1, 4, 5 and 7 are allowable over prior art as the closest prior arts are: Glaser et al. (US 20190378205 A1; hereafter referred to as Glaser), Olgiati (US 20190050629 A1; hereafter referred to as Olgiati), Flick et al. (US 20220004775 A1; hereafter referred to as Flick) and Biancale (US 20220004761 A1; hereafter referred to as Biancale). However, when looking at all he available prior arts none teaches: “determining, when the identified object disappears after the N1 images but reappears in less than or equal to N2 images of the plurality of images following the N1 images, that the identified object detected in the N2 images is a continued detection of the identified object detected in the N1 images; when the identified object is detected in N1 + N2 images, determining that the identified object remains in an area under observation, and determining that the identified object has left the area under observation when the identified object is not detected in N1 + N2 image”.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAISALI RAO KOPPOLU whose telephone number is (571)270-0273. The examiner can normally be reached Monday - Friday 8:30 - 5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
VAISALI RAO. KOPPOLU
Examiner
Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights