DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 5-9, 11 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xuhan (CN 116958756A) in view of Goel (U.S. PG-PUB NO. 2021/0181757).
-Regarding claim 1, Xuhan discloses a multi-task learning apparatus (abstract), comprising: a memory storing computer-executable instructions (storage unit 908, FIG. 9); and at least one processor configured to access the memory and execute the computer-executable instructions (computing unit 901, FIG. 9), wherein the at least one processor is configured to: perform a first task in which there is no ground truth for an input image in a multi-task to predict a result of the first task (S301, obtaining the reference prediction result of the first training sample, wherein the reference prediction result is not obtained based on the label of the first training sample, FIG. 3), and train the multi-task such that a loss function between the predicted result of the at least one task and ground truth of the at least one task (S303, based on the reference prediction result, the actual prediction result of the first training sample and the label of the first training sample, obtaining the total loss function, FIG. 3).
Xuhan is silent to teaching that perform at least one task in which there is ground truth with respect to a generation image generated by concatenating the predicted result of the first task and the input image to predict a result of the at least one task; minimizing loss function. However, the claimed limitation is well known in the art as evidenced by Goel.
In the same field of endeavor, Goel teaches perform at least one task in which there is ground truth with respect to a generation image generated by concatenating the predicted result of the first task and the input image to predict a result of the at least one task ( The same ROI used to crop and/or pool the data at cropping and/or pooling component 500 may be used to crop the image 120 and/or the instance segmentation 506, each or either of which may be upsampled and/or concatenated, paragraph 85); minimizing loss function (backpropagating the joint loss through the entire ML architecture such that parameters of the different components are altered to minimize the joint loss, paragraph 106).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Xuhan with the teaching of Goel in order to provide richer multi-model features for downstream task predictions by combining and concatenating early task predictions with the original input image.
-Regarding claim 2, the combination further discloses the at least one processor is configured to update a weight of each of networks for performing the at least one task and apply a trainable parameter for the at least one task to the generation image to predict the result of the at least one task (Goel, adjusting zero or more parameters of each component discussed herein to reduce the joint loss, paragraph 114).
-Regarding claim 3, the combination further discloses the trainable parameter includes a first parameter and a second parameter; and the at least one processor is configured to multiply an output of each of the networks for performing the at least one task by the first parameter and add the second parameter to the multiplied value to predict the result of the at least one task (Goel, a layer of the backbone may comprise a filter/kernel having one or more weights or a bias value associated therewith, paragraph 63).
-Regarding claim 5, the combination further discloses the at least one processor is configured to: train the multi-task using a cross entropy loss, when the at least one task is a task for performing segmentation; and train the multi-task using a mean square error, when the at least one task is a task for detecting depth (Xuhan, MSE (Mean-Square Error), FIG. 1; Goel, a pixel-wise loss for classifying each pixel into one of the semantic classes (for the semantic segmentation data), paragraph 110).
-Regarding claim 6, the combination further discloses the at least one processor is configured to train the multi-task such that a loss function between a predicted result of the at least one task for the input image and the ground truth of the at least one task and the loss function between the predicted result of the at least one task for the generation image and the ground truth of the at least one task are minimized (Xuhan, S303, based on the reference prediction result, the actual prediction result of the first training sample and the label of the first training sample, obtaining the total loss function, FIG. 3).
-Regarding claim 7, Xuhan discloses a multi-task learning method (abstract), comprising: performing a first task in which there is no ground truth for an input image in a multi-task to predict a result of the first task (S301, obtaining the reference prediction result of the first training sample, wherein the reference prediction result is not obtained based on the label of the first training sample, FIG. 3); and training the multi-task such that a loss function between the predicted result of the at least one task and ground truth of the at least one task (S303, based on the reference prediction result, the actual prediction result of the first training sample and the label of the first training sample, obtaining the total loss function, FIG. 3).
Xuhan is silent to teaching that perform at least one task in which there is ground truth with respect to a generation image generated by concatenating the predicted result of the first task and the input image to predict a result of the at least one task; minimizing loss function. However, the claimed limitation is well known in the art as evidenced by Goel.
In the same field of endeavor, Goel teaches perform at least one task in which there is ground truth with respect to a generation image generated by concatenating the predicted result of the first task and the input image to predict a result of the at least one task ( The same ROI used to crop and/or pool the data at cropping and/or pooling component 500 may be used to crop the image 120 and/or the instance segmentation 506, each or either of which may be upsampled and/or concatenated, paragraph 85); minimizing loss function (backpropagating the joint loss through the entire ML architecture such that parameters of the different components are altered to minimize the joint loss, paragraph 106).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Xuhan with the teaching of Goel in order to provide richer multi-model features for downstream task predictions by combining and concatenating early task predictions with the original input image.
-Regarding claim 8, the combination further discloses predicting the result of the at least one task includes updating a weight of each of networks for performing the at least one task and applying a trainable parameter for the at least one task to the generation image to predict the result of the at least one task (Goel, adjusting zero or more parameters of each component discussed herein to reduce the joint loss, paragraph 114).
-Regarding claim 9, the combination further discloses the trainable parameter includes a first parameter and a second parameter; and predicting the result of the at least one task includes multiplying an output of each of the networks for performing the at least one task by the first parameter and adding the second parameter to the multiplied value to predict the result of the at least one task (Goel, a layer of the backbone may comprise a filter/kernel having one or more weights or a bias value associated therewith, paragraph 63).
-Regarding claim 11, the combination further discloses training the multi-task includes: training the multi-task using a cross entropy loss, when the at least one task is a task for performing segmentation; and training the multi-task using a mean square error, when the at least one task is a task for detecting depth (Xuhan, MSE (Mean-Square Error), FIG. 1; Goel, a pixel-wise loss for classifying each pixel into one of the semantic classes (for the semantic segmentation data), paragraph 110).
-Regarding claim 12, the combination further discloses training the multi-task includes training the multi-task such that a loss function between a predicted result of the at least one task for the input image and the ground truth of the at least one task and the loss function between the predicted result of the at least one task for the generation image and the ground truth of the at least one task are minimized (Xuhan, S303, based on the reference prediction result, the actual prediction result of the first training sample and the label of the first training sample, obtaining the total loss function, FIG. 3).
Allowable Subject Matter
Claims 4 and 10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 13 is allowed.
Response to Arguments
Applicant's arguments filed 7/29/2026 have been fully considered but they are not persuasive.
1. "First task in which there is no ground truth for an input image"
Applicant argues that Xuhan only teaches a reference prediction obtained without using an available label, which differs from a task having no ground truth for the input image. Applicant further argues that Goel does not show that its instance segmentation has no ground truth for a particular input image.
The Examiner respectfully disagrees. Goel expressly teaches that ground truth is not available for every task for every training image. Goel explains that training images may be labeled with ground truth for ROI and semantic classification but not for instance segmentation, depth, direction, and/or three-dimensional ROI (Goel, paragraph 101). Goel's training data is organized into batches, each associated with different ground truth (paragraph 102).
Goel also teaches that a given image is not associated with ground truth for each component of the ML architecture that generates an output. Losses are therefore computed only from the subset of outputs for which ground truth is available (paragraph 108). Goel further contemplates semi-supervised training in which only a subset of data is labeled (paragraph 103).
Accordingly, for an input image lacking instance segmentation ground truth, Goel's instance segmentation component 502 performs "a first task in which there is no ground truth for an input image in a multi-task," and outputs a predicted result, namely instance segmentation 506 (paragraph 83, 97). Goel expressly refers to each output of its jointly trained architecture as a "task" (paragraph 19, 37).
Xuhan is consistent with this teaching. Xuhan obtains predictions for unlabeled second training samples, i.e., samples with no ground truth, and uses those predictions in the total loss (Xuhan, S405–S410, FIG. 5).
2. "Generation image generated by concatenating the predicted result of the first task and the input image"
Applicant argues that Goel merely concatenates ROI-cropped data with semantic segmentation, direction, and depth data to generate a three-dimensional ROI.
The Examiner respectfully disagrees. Goel expressly teaches concatenating the image 120 and the instance segmentation 506 (at 514) and providing the concatenated data to the three-dimensional ROI component 512 (paragraph 85). Goel likewise states that the 3D ROI is determined by concatenating an image crop associated with the object and the instance segmentation (paragraph 25). The concatenated data is thus generated by concatenating (i) the predicted result of the first task (instance segmentation 506) and (ii) the input image (image 120).
3. "At least one task in which there is ground truth" performed on the generation image, and training to minimize the corresponding loss
Applicant argues that Goel paragraph 106 discloses only a generic joint loss, not a loss between the predicted result of the task performed on the generation image and that task's ground truth.
The Examiner respectfully disagrees. The three-dimensional ROI component 512 performs a task, 3D ROI determination, on the concatenated data and outputs 3D ROI 516 (paragraphs 85–86). Goel computes task-specific losses based on errors between each output and its respective ground truth (paragraph 20). These include classification and regression losses with respect to the two- and/or three-dimensional ROIs (paragraph 110), computed where ground truth for that task is available (paragraph 108). The task-specific losses are summed into the joint loss (paragraphs 109-111). The joint loss is backpropagated so that the component parameters are altered to minimize it (paragraphs 106, 114). Minimizing a joint loss that includes the 3D ROI loss necessarily trains the multi-task to minimize the loss between the predicted result of the at least one task and that task's ground truth. Goel's joint loss is therefore not merely "generic." It expressly includes the loss between the 3D ROI predicted from the concatenated image/instance-segmentation data and the 3D ROI ground truth.
4. Motivation to combine and hindsight
Applicant argues that the "richer features" rationale does not explain the particular claimed arrangement and that the rejection relies on impermissible hindsight.
The Examiner respectfully disagrees. In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). In this case, the claimed arrangement is not reconstructed from Applicant's disclosure. It is the arrangement Goel itself describes: a task output lacking ground truth for some training images (paragraphs 101, 108) is concatenated with the input image (paragraphs 25, 85); the result is fed to a downstream task that has ground truth (paragraph 86, 110); the network is jointly trained to minimize the resulting loss (paragraphs 106, 114). Goel also supplies the reason for doing so: improving the accuracy and amount of information in object detection within a single jointly trained model (paragraphs 17-19, 26). Combining Xuhan's multi-task training framework with Goel's architecture is the use of a known technique to improve a similar system in the same way. It yields predictable results. KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 417 (2007). Additionally, in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
5. Claim 7 and the dependent claims
Applicant relies on the arguments for claim 1 with respect to claim 7. Applicant presents no separate arguments for dependent claims 2, 3, 5, 6, 8, 9, 11, and 12 beyond their dependency. Those arguments are unpersuasive for the reasons above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PING Y HSIEH whose telephone number is (571)270-3011. The examiner can normally be reached Monday-Friday, 9am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PING Y HSIEH/Primary Examiner, Art Unit 2664
Prosecution Insights