Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 2023-185157, filed on October 30, 2023 with Japanese Patent Office. However, the priority documents have not been received.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/23/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 5, and 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 20240161303 A1) referred to as Kim hereinafter and further in view of Tsunoda (US 20230245427 A1).
Regarding claim 1, Kim teaches An attribute identification device comprising: (“in a method for identifying an object executed by an apparatus for identifying an object,” Kim, para. [0015]) a memory configured to store instructions; a processor configured to execute the instructions to: (“a memory storing one or more programs; and a processor executing the stored one or more programs” Kim, para. [0023])
when given a frame, derive a bounding box of a detection target object from within the frame based on a learning model that includes multiple layers, (“an apparatus for auto segmentation using a bounding box may be provided, the apparatus comprising a database storing a first object image including an object labeled with a bounding box, which is a pre-learning target” Kim, para. [0023]) and define output data of one layer in the learning model or the frame itself as a feature value of the frame; (“the mask prediction module 150 performs segmentation by using the controller of each object region calculated through the detection branch module 130 as a parameter and predicts N segmentation masks. The mask prediction module 150 outputs the prediction masks. The prediction masks may be used for learning the segmentation model.” Kim, para. [0115])
acquire a feature value of the bounding box of an attribute identification target object from the feature value of the frame; (“the apparatus for auto segmentation restricts a target object to always exist only inside the bounding box. The apparatus for auto segmentation restricts the width and height obtained through predicted values from segmentation not to be greater than the given bounding box.” Kim, para. [0060])
and extract one or more attributes of the attribute identification target object from the bounding box of the attribute identification target object; (“The apparatus for auto segmentation extracts a segmentation result from new data (e.g., a second object image including an object to be identified) by utilizing a segmentation model trained using the first object image labeled only with the bounding box.” Kim, para. [0058])
and a storage device that stores a combination of a frame ID, the tracking ID, the feature value of the bounding box of the attribute identification target object, and the one or more attributes of the attribute identification target object; (“an apparatus for auto segmentation using a bounding box may be provided, the apparatus comprising a database storing a first object image including an object labeled with a bounding box, which is a pre-learning target; a memory storing one or more programs; and a processor executing the stored one or more programs, wherein the processor is configured to receive a first object image including an object labeled with a bounding box, which is a pre-learning target, learn a segmentation model by classifying an object and a background from the bounding box of the received first object image, and segment an object from a second object image, which is an identification target, using the learned segmentation model.” Kim, para. [0023])
wherein the processor determines a similarity degree between the feature value of the bounding box of the attribute identification target object and the feature value of a past bounding box corresponding to the tracking ID of the bounding box, (“Utilizing the color similarity as a threshold, the segmentation model learns whether each edge should be positive or negative. FIGS. 4 and 5 illustrate examples of the operation of calculating a first loss and a second loss according to another embodiment of the present disclosure. The apparatus for auto segmentation according to one embodiment of the present disclosure uses a first loss and a second loss during the process of learning a segmentation model only through a bounding box. Here, the first loss may be a projection loss, and the second loss may be a pixel pairwise loss. Regarding the projection loss, FIG. 4 shows object image 201 as a learning target, image 202 from the projection of a predicted mask, and image 203 from the project of a bounding box. The apparatus for auto segmentation restricts the predicted segmentation mask not to leave the bounding box given together during learning. In other words, the apparatus for auto segmentation prevents the prediction of a segmentation model from leaving the bounding box area in which an object is expected to exist. Regarding the pixel pairwise loss, FIG. 5 shows a pixel pairwise relationship 301 between each pixel and its neighboring pixels and eight consistency maps 302, 303. In what follows, the pixel pairwise loss is described. Based on the coordinates (i,j) and (l,k), a segmentation model may predict the probability that pixels at the corresponding coordinates of the prediction masks, which are prediction values, belong to the foreground. The corresponding values, m.sub.(i,j) and m.sub.(l,k) represent the probabilities predicted by the segmentation model that pixels at the corresponding coordinates belong to the foreground (where the foreground includes an object in this case). The probability that the coordinates (i,j) and (l,k) have the same label is P(y.sub.e=1), Kim, paras. [0090]-[0095]) and figs. 4 and 5.
However, Kim does not teach add a tracking ID to the bounding box; and when the similarity degree is equal to or greater than a predetermined threshold, stops extracting the one or more attributes of the attribute identification target object,
Tsunoda teaches add a tracking ID to the bounding box; (“The two-stage method is a method of performing tracking with use of a detector and a feature extractor which are independent from each other. In the two-stage method, first, the tracking apparatus performs detection with respect to each frame of a moving image with use of the detector, and obtains a circumscribed rectangle (bounding box (hereinafter referred to as a “BBox”)) of a tracking target object obtained as a result of detection.” Tsunoda, para. [0021])
and when the similarity degree is equal to or greater than a predetermined threshold, stops extracting the one or more attributes of the attribute identification target object, (“in a second exemplary embodiment, the tracking apparatus 200 performs filtering only in a case where the degree of overlapping of persons is greater than or equal to a fixed threshold value, and does not perform filtering in a case where the degree of overlapping of persons is less than the fixed threshold value.” Tsunoda, para. [0108])
and identifies the one or more attributes of the attribute identification target object in the bounding box by diverting the one or more attributes corresponding to the feature value of the past bounding box. (“The tracking apparatus performs ID assignment, which associates a person acquired from each BBox of the current frame 103 with a track for up to the previous frame, based on the proximity of feature quantities. Then, the tracking apparatus assigns the ID of the person 104b to a person acquired from the BBox 105c in the frame 103 and assigns the ID of the person 104c to a person acquired from the BBox 105b in the frame 103” Tsunoda, para. [0029])
Kim and Tsunoda are combinable because they are from the same field of endeavor, object similarity.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim in light of Tsunoda’s adding tracking ID to the bounding box. One would have been motivated to do so because it can increase accuracy of image recognition. (Tsunoda, para. [0096])
Regarding claim 2, Kim does not teach wherein when the multiple layers in the learning model are divided into a first half and a second half, the processor defines output data of the first half layer as the feature value of the frame.
Tsunoda teaches wherein when the multiple layers in the learning model are divided into a first half and a second half, the processor defines output data of the first half layer as the feature value of the frame. (“A middle feature (m-feat) 602 is an input to the RNN 601, and, as mentioned above, has a size of [[16, 32, 64], [8, 16, 128]]. When being input to the first FC of the RNN 601, the middle feature 602 is made flat. Thus, the middle feature 602 has an input size of [49152]. The second FC, the Cony, the reshape layer, and the resize layer of the RNN 601 perform conversion into an appropriate size.” Tsunoda, para. [0060]) and fig. 4
Kim and Tsunoda are combinable because they are from the same field of endeavor, object similarity.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim in light of Tsunoda’s multiple layers divided in half. One would have been motivated to do so because it can increase accuracy of image recognition. (Tsunoda, para. [0096])
Regarding claim 5, Kim does not teach wherein when determining the similarity degree between the feature value of the bounding box of the attribute identification target object and the feature value of the past bounding box corresponding to the tracking ID of the bounding box, the processor obtains cosine similarity degree based on conversion results of converting the two feature values respectively into a vector with a predetermined number of elements, uses the cosine similarity degree as the similarity degree between the two feature values.
Tsunoda teaches wherein when determining the similarity degree between the feature value of the bounding box of the attribute identification target object and the feature value of the past bounding box corresponding to the tracking ID of the bounding box, the processor obtains cosine similarity degree based on conversion results of converting the two feature values respectively into a vector with a predetermined number of elements, uses the cosine similarity degree as the similarity degree between the two feature values. (“The cost calculation unit 205 (FIG. 2A) calculates the degree of similarity between the filtered feature 607 obtained by the above-mentioned calculation and a filtered feature already included in the track and converts the calculated degree of similarity into a cost. Here, the cost calculation unit 205 calculates a cosine similarity as the degree of similarity. The cosine similarity takes real number values of −1 to +1, where +1 indicates most similar. Usually, an evaluation value which makes positive sense as the value is larger is called a “score”, and the opposite evaluation value thereof is called a “cost”. To perform conversion into an evaluation value which indicates a high degree of similarity as the value is smaller, i.e., a cost, the cost calculation unit 205 multiplies the cosine similarity by −1. Moreover, depending on an algorithm for an assignment problem which is used in next step S306, it is desirable that the cost be 0 or more. Therefore, here, the cost calculation unit 205 adds 1 as a bias to the cost. Performing the above-described calculation for the number of combinations of the detection objects Det1 to Det3 and the tracks Track1 to Track4 causes the cost matrix 501 to be calculated.” Tsunoda, para. [0051])
Kim and Tsunoda are combinable because they are from the same field of endeavor, object similarity.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim in light of Tsunoda’s cosine similarity degree. One would have been motivated to do so because it can increase accuracy of image recognition. (Tsunoda, para. [0096])
Regarding claim 6, Kim does not teach wherein when determining the similarity degree between the feature value of the bounding box of the attribute identification target object and the feature value of the past bounding box corresponding to the tracking ID of the bounding box, the processor obtains cosine similarity degree based on conversion results of converting the two feature values respectively into a vector with a predetermined number of elements, uses the cosine similarity degree as the similarity degree between the two feature values.
Tsunoda teaches wherein when determining the similarity degree between the feature value of the bounding box of the attribute identification target object and the feature value of the past bounding box corresponding to the tracking ID of the bounding box, (“Since the detection objects Det2 and Det3 have degrees of overlapping of human bodies with each other greater than or equal to the threshold value, as with the first exemplary embodiment, the cost calculation unit 205 calculates a filtered feature (f-feat) 1005 with use of the filter 1022 and the FC 1023 illustrated in FIG. 10B. Then, as with the first exemplary embodiment, the cost calculation unit 205 calculates a cosine similarity between the calculated filtered feature (f-feat) 1005 and a filtered feature (f-feat) included in the track, and converts the cosine similarity into a cost. Here, the cost calculation unit 205 calculates costs other than the costs 507, 508, 509, and 510, included in the cost matrix 506.” Tsunoda, para. [0119])
the processor obtains cosine similarity degree based on conversion results of converting the two feature values respectively into a vector with a predetermined number of elements, uses the cosine similarity degree as the similarity degree between the two feature values. (“the GAP 1012 performs processing for performing, with respect to respective layer features different in spatial resolution, global average pooling for each layer and interconnecting respective results of the processing. Thus, the input of the FC 1013 is the same as an input from a filter processing unit 604 illustrated in FIG. 6A to the FC, and is a 704-dimensional vector.” Tsunoda, para. [0074])
Kim and Tsunoda are combinable because they are from the same field of endeavor, object similarity.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim in light of Tsunoda’s multiple layers divided in half. One would have been motivated to do so because it can increase accuracy of image recognition. (Tsunoda, para. [0096])
Regarding claim 7, refer to the explanation of claim 1.
Regarding claim 8, Kim teaches A non-transitory computer-readable recording medium in which an attribute identification program is stored, wherein the attribute identification program causes a computer to execute: (“when the processor executes a method, a non-transitory computer-readable storage medium may be provided for storing instructions used by the processor to execute the method,” Kim, para. [0138])
Regarding rest of claim 8, refer to the explanation of claim 1.
Allowable Subject Matter
Claims 3 and 4 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claims 3 and 4, none of the cited prior arts teach or suggest, alone or in combination, the particular combination of steps or elements as recited in the independent claims. Specifically, the combination of the cited prior arts does not specifically disclose, teach, or suggest as a whole the limitation “the processor obtaining CKA (Centered Kernel alignment) based on conversion results of converting the two feature values respectively into a matrix with a predetermined number of rows, uses the CKA as the similarity degree between the two feature values.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PARDIS SOHRABY whose telephone number is (571)270-0809. The examiner can normally be reached Monday - Friday 9 am till 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PARDIS SOHRABY/Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights