DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The Preliminary Amendment filed 22 November 2024 has been received and considered.
The IDS’s filed 30 May 2025 and 28 July 2026 have been received and considered.
The Priority Documents electronically retrieved 02 January 2025 have been considered.
Claims 1 – 13 are pending.
Claim 12 has been preliminarily amended.
Claims 1 – 13, all of the claims pending in this application, have been rejected.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a matching module configured to match” in claim 2.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
In this situation, the applied prior art appears likely to have partially common authorship/inventorship with this application. In appropriate fact situations, Applicant may rely on the exception under 35 U.S.C. 102(b)(1)(A) to overcome this rejection under 35 U.S.C. 102(a)(1) by a showing under 37 CFR 1.130(a) that the subject matter disclosed in the reference was obtained directly or indirectly from the inventor or a joint inventor of this application, and is therefore not prior art under 35 U.S.C. 102(a)(1). Alternatively, applicant may rely on the exception under 35 U.S.C. 102(b)(1)(B) by providing evidence of a prior public disclosure via an affidavit or declaration under 37 CFR 1.130(b).
Claims 1 – 13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Non-Patent Literature “Joint Appearance and Motion Model With Temporal Transformer for Multiple Object Tracking” to Kim et al. (hereinafter Kim).
Claim 1
Regarding Claim 1, an independent method claim, Kim teaches an operating method of an electronic device, comprising: detecting at least one object in a frame at a current time point (Figure 2, I^(t); "We develop two parallel modules, 1) the ReID embedding module and 2) the motion estimation module, to associate previously tracked objects with detected objects in the current frame.", I. INTRODUCTION);
PNG
media_image1.png
362
631
media_image1.png
Greyscale
obtaining, using a neural network model (Figure 2), a matching result between the at least one detected object and at least one object tracked in frames at a previous time point that precedes the current time point, based on information about the at least one detected object, information about the at least one tracked object, a frame feature of the frame, and an object-recognized frame feature of the previous time point ("The ReID embedding module extracts ReID features of the detected objects, while the motion estimation module predicts expected locations of the previously tracked objects in the current frame. The motion-guided association then matches the tracked objects with the detected objects using both ReID features and predicted locations.", I. INTRODUCTION; and
obtaining trajectory information of the at least one detected object up to the current time point, based on the matching result ("Temporal transformer conveys the feature information of tracked objects in the trajectory set O(t−1) at the previous frame into the current frame feature F(t) to obtain an object-aware frame feature F(t).", III. PROPOSED NETWORK - B. TEMPORAL TRANSFORMER).
Claim 2
Regarding Claim 2, dependent on claim 1, Kim teaches the invention as claimed in claim 1.
Kim further teaches wherein the neural network model comprises: a transformer configured to output an object-recognized frame feature of the current time point, based on the frame feature and the object-recognized frame feature of the previous time point ("Temporal transformer conveys the feature information of tracked objects in the trajectory set O(t−1) at the previous frame into the current frame feature F(t) to obtain an object-aware frame feature F(t). For this purpose, we apply RoIAlign to tracked object boxes in O^(t−1) on the object-aware frame feature at the previous frame to obtain object features…the information of tracked objects can be effectively interfused to the current frame.", III. PROPOSED NETWORK - B. TEMPORAL TRANSFORMER);
a re-identification (ReID) embedding module configured to output a ReID feature of the at least one detected object, based on the object-recognized frame feature of the current time point and the object-recognized frame feature of the previous time point ("Figure 4 shows the structure of the ReID embedding head, which provides ReID features of the detected objects. It includes MCA to explore features of tracked object in O^(t−1) effectively. We interfuse features of tracked objects to those of detected objects in the current frame.", III. PROPOSED NETWORK - C. ReID EMBEDDING MODULE);
a motion estimation module configured to output a heatmap comprising information about a position estimated at the current time point for each of the at least one tracked object, based on the object-recognized frame feature of the current time point and the object-recognized frame feature of the previous time point (Figure 5, H(t); "The motion estimation module, in Figure5, finds positions of objects in the previous trajectories O^(t−1) at the current frame." III. PROPOSED NETWORK - D. MOTION ESTIMATION MODULE); and
PNG
media_image2.png
297
347
media_image2.png
Greyscale
a matching module configured to match the at least one detected object and the at least one tracked object, based on the information about the at least one tracked object, the information about the at least one detected object, the ReID feature, and the heatmap ("The proposed motion-guided association performs matching between tracked objects in O^(t−1) and detected objects in B^(t) using both ReID features and motion-based tracking locations.", III. PROPOSED NETWORK - E. MOTION-GUIDED ASSOCIATION).
Claim 3
Regarding Claim 3, dependent on claim 2, Kim teaches the invention as claimed in claim 2.
Kim further teaches wherein the transformer is configured to: obtain first feature matrices based on the object-recognized frame feature of the previous time point (Figure 3; "Then, we concatenate those tracked object features to compose a feature matrix ((R sub O)^(t-1))…", III. PROPOSED NETWORK - B. TEMPORAL TRANSFORMER);
PNG
media_image3.png
254
347
media_image3.png
Greyscale
obtain a second feature matrix based on the frame feature (Figure 3, ((F sub Q)^(t))); and
PNG
media_image4.png
254
347
media_image4.png
Greyscale
output the object-recognized frame feature of the current time point, based on the first feature matrices, the second feature matrix, and the frame feature (Figure 3, ((F sub O)^(t))).
PNG
media_image5.png
254
347
media_image5.png
Greyscale
Claim 4
Regarding Claim 4, dependent on claim 3, Kim teaches the invention as claimed in claim 3.
Kim further teaches wherein the transformer is configured to: obtain a first object feature by performing a region-of-interest alignment (ROI Align) on the at least one tracked object on the object-recognized frame feature of the previous time point (For this purpose, we apply RoIAlign [41] φ(·) to tracked object boxes in O^(t−1) on the object aware frame feature at the previous frame to obtain object features.", III. PROPOSED NETWORK - B. TEMPORAL TRANSFORMER);
and obtain the first feature matrices based on the first object feature ("Then, we concatenate those tracked object features to compose a feature matrix…", III. PROPOSED NETWORK - B. TEMPORAL TRANSFORMER).
Claim 5
Regarding Claim 5, dependent on claim 3, Kim teaches the invention as claimed in claim 3.
Kim further teaches wherein the transformer is configured to: obtain a fused object feature from the first feature matrices and the second feature matrix, based on a cross-attention layer ("Then, we obtain the interfused object feature through multi-head cross attention(MCA)…Based on the affinity between, the information of tracked objects can be effectively interfused to the current frame.", III. PROPOSED NETWORK - B. TEMPORAL TRANSFORMER); and
output the object-recognized frame feature of the current time point, based on the fused object feature and the frame feature (Rejected as applied directly above).
Claim 6
Regarding Claim 6, dependent on claim 2, Kim teaches the invention as claimed in claim 2.
Kim further teaches wherein the ReID embedding module is configured to: obtain first feature matrices based on the object-recognized frame feature of the previous time point (Rejected as applied to claim 2);
obtain a third feature matrix based on the object-recognized frame feature of the current time point (Figure 4, ((R sub Q)^(t))); and
PNG
media_image6.png
231
347
media_image6.png
Greyscale
output the ReID feature of the at least one detected object, based on the first feature matrices and the third feature matrix (Figure 4).
PNG
media_image7.png
231
347
media_image7.png
Greyscale
Claim 7
Regarding Claim 7, dependent on claim 6, Kim teaches the invention as claimed in claim 6.
Kim further teaches wherein the ReID embedding module is configured to: obtain a second object feature by performing ROI Align on the at least one detected object on the object-recognized frame feature of the current time point (Figure 4); and
PNG
media_image8.png
230
343
media_image8.png
Greyscale
obtain the third feature matrix based on the second object feature (Figure 4).
PNG
media_image9.png
230
343
media_image9.png
Greyscale
Claim 8
Regarding Claim 8, dependent on claim 2, Kim teaches the invention as claimed in claim 2.
Kim further teaches wherein the motion estimation module is configured to: obtain a third object feature based on the object-recognized frame feature of the previous time point (Figure 5);
PNG
media_image10.png
296
352
media_image10.png
Greyscale
obtain a fourth object feature based on the object-recognized frame feature of the current time point (Figure 5); and
PNG
media_image11.png
296
352
media_image11.png
Greyscale
output the heatmap based on the third object feature and the fourth object feature (Figure 5).
PNG
media_image12.png
296
352
media_image12.png
Greyscale
Claim 9
Regarding Claim 9, dependent on claim 8, Kim teaches the invention as claimed in claim 8.
Kim further teaches wherein the motion estimation module is configured to: obtain the fourth object feature by performing ROI Align on a search region on the object-recognized frame feature of the current time point (Figure 5),
PNG
media_image13.png
296
352
media_image13.png
Greyscale
wherein the search region is obtained by adjusting a scale of a bounding box of the at least one detected object ("For each tracked object, we set a search region by expanding the object box by scale factor 3 while maintaining its center position and find a location of the corresponding object within the search region at the current frame.", III. PROPOSED NETWORK - D. MOTION ESTIMATION MODULE).
Claim 10
Regarding Claim 10, dependent on claim 2, Kim teaches the invention as claimed in claim 2.
Kim further teaches wherein the matching module is configured to: calculate a first similarity between the at least one tracked object and the at least one detected object, based on the ReID feature ("First, we compute the appearance similarity between the ReID features f (t−1), respectively…Formula (8)", III. PROPOSED NETWORK - E. MOTION-GUIDED ASSOCIATION);
calculate a second similarity between the at least one tracked object and the at least one detected object, based on the information about the position estimated at the current time point for each of the at least one tracked object, the information comprised in the heatmap ("We compute the motion-based similarity between ok(t−1) and bi(t) based on intersection over union (IoU)…Formula (11)", III. PROPOSED NETWORK - E. MOTION-GUIDED ASSOCIATION); and
output the matching result based on a weighted sum of the first similarity and the second similarity ("Finally, the association similarity is obtained by the weighted sum of appearance and motion-based similarity…Formula (12)", III. PROPOSED NETWORK - E. MOTION-GUIDED ASSOCIATION).
Claim 11
Regarding Claim 11, dependent on claim 10, Kim teaches the invention as claimed in claim 10.
Kim further teaches wherein the first similarity is based on a bidirectional softmax similarity and cosine similarity between a ReID feature of the current time point and a ReID feature of the previous time point ("where Sb is the bi-directional softmax similarity…Formula (9); "Also, the cosine similarity Sc is defined as…Formula 10", III. PROPOSED NETWORK - E. MOTION-GUIDED ASSOCIATION).
Claim 12
Regarding Claim 12, an independent non-transitory computer-readable storage medium claim, Kim teaches a non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the operating method claim 1 (Rejected as applied to claim 1).
Claim 13, an independent device claim, is rejected for the same reasons as applied to the above claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Non-Patent Literature “MeMOT:Multi-Object Tracking with Memory” to Cai et al. (hereinafter Cai). https://arxiv.org/pdf/2203.16761v1
Non-Patent Literature “TrackFormer: Multi-Object Tracking with Transformers” to Meinhardt et al. (hereinafter Meinhardt). https://arxiv.org/pdf/2101.02702
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ronde Miller whose telephone number is (703) 756-5686 The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Gregory Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RONDE LEE MILLER/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights