DETAILED ACTION
Reissue
The present reissue application is directed to US 11,645,869 B2 (“869 Patent”). 869 Patent issued on May 9, 2023 with claims 1-20 from application 16/808,357 filed on March 3, 2020, which is a continuation of parent application 16/024,823 filed on June 30, 2018, which is a continuation of application 15/224,487 filed on July 29, 2016, and claims priority to 62/342,945 filed on May 28, 2016.
This application was filed on May 7, 2025. Since this date is after September 16, 2012, all references to 35 U.S.C. 251 and 37 CFR 1.172, 1.175, and 3.73 are to the current provisions. Furthermore, the present application is being examined under the first inventor to file provisions of the AIA .
This application presents broadened claims, which are permitted because Applicant filed these claims and demonstrated an intent to broaden within two years of the issue date of 869 Patent.
The most recent amendment was filed on May 7, 2025. The status of the claims is:
Claims 1-4, 7-14, and 17-20: Amended
Claims 5, 6, 15, and 16: Original
Claims 21-32: New
This is a first, non-final action.
References and Documents Cited in this Action
869 Patent (US 11,645,869 B2)
Choi (US 2017/0124415 A1)
Wolf (US 2019/0042892 A1)
US 10,032,067 B2
Summary of Rejections and Objections in this Action
Claims 1-32 are rejected as being based upon a defective reissue declaration under 35 U.S.C. 251.
Claims 21-32 are rejected under 35 U.S.C. 251 as being an impermissible recapture
Claims 21, 23-26, and 28-32 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Choi.
Claims 22 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Wolf.
Claim 11 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. US 10,032,067 B2.
Summary of the Claims
869 is directed to a system and method for recognizing objects in an image. Claims 1, 11, 21, 26, 31, and 32 are the independent claims. Claim 21 is representative:
21. An apparatus to detect an object in an image, the apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, cause the
processor to perform operations comprising:
receiving as an input a feature of a detected object;
determining a regression loss associated with parameters based on the feature of the detected object;
determining a classification loss for a classification score for the feature of the detected object; and
generating a multi-task loss based on the regression loss and the classification loss.
Oath/Declaration
The reissue oath/declaration filed with this application is defective (see 37 CFR 1.175 and MPEP § 1414). Applicant’s statement of an error upon which this reissue is based is insufficient:
PNG
media_image1.png
154
554
media_image1.png
Greyscale
The declaration does not sufficiently describe an error upon which this reissue is based. For an application filed on or after September 16, 2012 that seeks to enlarge the scope of the claims of the patent, the reissue oath or declaration must identify a claim that the application seeks to broaden in the identification of the error that is relied upon to support the reissue application. A general statement, e.g., that all claims are broadened, is not sufficient to satisfy this requirement. In specifically identifying the error as required by 37 CFR 1.175(a), it is sufficient that the reissue oath/declaration identify the claim being broadened and a single word, phrase, or expression in the specification or in an original claim, and how it renders the original patent wholly or partly inoperative or invalid. For example, Applicant may state that claim 1 is broadened and identify a word or phrase in claim 1 that is not recited in the new claims.
Applicant must submit a new reissue declaration (rather than merely correct the error statement in remarks) because no proper reissue declaration has been yet entered in this reissue application.
Claim Rejections - 35 USC § 251
Claims 1-32 are rejected as being based upon a defective reissue declaration under 35 U.S.C. 251 as set forth above. See 37 CFR 1.175.
The nature of the defect(s) in the declaration is set forth in the discussion above in this Office action.
Claims 21-32 are also rejected under 35 U.S.C. 251 as being an impermissible recapture of broadened claimed subject matter surrendered in the application for the patent upon which the present reissue is based. In re McDonald, 43 F.4th 1340, 1345, 2022 USPQ2d 745 (Fed. Cir. 2022); Greenliant Systems, Inc. et al v. Xicor LLC, 692 F.3d 1261, 103 USPQ2d 1951 (Fed. Cir. 2012); In re Youman, 679 F.3d 1335, 102 USPQ2d 1862 (Fed. Cir. 2012); In re Shahram Mostafazadeh and Joseph O. Smith, 643 F.3d 1353, 98 USPQ2d 1639 (Fed. Cir. 2011); North American Container, Inc. v. Plastipak Packaging, Inc., 415 F.3d 1335, 75 USPQ2d 1545 (Fed. Cir. 2005); Pannu v. Storz Instruments Inc., 258 F.3d 1366, 59 USPQ2d 1597 (Fed. Cir. 2001); Hester Industries, Inc. v. Stein, Inc., 142 F.3d 1472, 46 USPQ2d 1641 (Fed. Cir. 1998); In re Clement, 131 F.3d 1464, 45 USPQ2d 1161 (Fed. Cir. 1997); Ball Corp. v. United States, 729 F.2d 1429, 1436, 221 USPQ 289, 295 (Fed. Cir. 1984). The reissue application contains claim(s) that are broader than the issued patent claims. The record of the application for the patent family shows that the broadening aspect (in the reissue) relates to claimed subject matter that applicant previously surrendered during the prosecution of the application. Accordingly, the narrow scope of the claims in the patent was not an error within the meaning of 35 U.S.C. 251, and the broader scope of claim subject matter surrendered in the application for the patent cannot be recaptured by the filing of the present reissue application.
This rejection is based on the following three-step test for recapture. See MPEP 1412.02. In the discussion below, “reissue claims” refers to pending claims of this reissue application; “patent claims” refers to claims issued in 869 Patent; and “original claims” refers to claims that were presented in the original application prior to surrender, wherein the “original application” includes the prosecution record of the application that issued as 869 Patent as well as the patent family’s entire prosecution history. See In re Youman, 679 F.3d 1335, 1346 n.4, 102 USPQ2d 1862, 1870 n.4 (Fed. Cir. 2012) and MBO Laboratories, Inc. v. Becton, Dickinson & Co., 602 F.3d 1306, 1316-17, 94 USPQ2d 1598 (Fed. Cir. 2010).
1) Was there broadening?
Yes, the reissue claims are broader than the patent claims. For example, patent claim 1 recites “wherein the loss generator receives an output from the alignment network of the alignment regression loss and an output from the first classification network of the classification loss” and patent claim 11 recites “wherein the loss generator receives an output from the alignment network of the alignment regression loss and an output from the detection box network of the bounding-box regression loss.” These limitations are not in new reissue claims 21-32.
2) Does any broadening aspect of the reissue claim relate to surrendered subject matter?
Yes, a broadening aspect of the reissue claims relates to surrendered subject matter. During the prosecution of 869 Patent (16/808,357), Applicant added “wherein the loss generator receives an output from the alignment network of the alignment regression loss and an output from the first classification network of the classification loss” to independent claim 1 and “wherein the loss generator receives an output from the alignment network of the alignment regression loss and an output from the detection box network of the bounding-box regression loss” to independent claim 11 (see claim amendments filed on November 18, 2021 and September 2, 2022 in 16/808,357). These additional limitations were added to overcome rejections over the prior art. For example, Applicant argued that the Wolf reference cited by the examiner “does not disclose a loss generator that ‘receives an output from the alignment network and an output from the classification network” (see response filed on November 18, 2021 in 16/808,357) and that “Wolf does not appear to disclose a loss generator that receives and operates on two outputs from two networks as recited in claim 1” (see response filed on September 2, 2022 in 16/808,357). Therefore, these limitations are surrendered subject matter.
3) Were the reissue claims materially narrowed in other aspects such that recapture is avoided?
No, the reissue claims are not materially narrowed such that recapture is avoided. Reissue claims 21-32 lack the surrendered subject matter entirely.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 21, 23-26, and 28-32 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Choi.
Regarding independent claim 21, Choi discloses an apparatus to detect an object in an image (Figures 1-3), the apparatus comprising:
a processor 604 (Figure 5; paragraphs [0067]-[0073]); and
a memory 608 (Figure 5; paragraphs [0067]-[0073]) storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
receiving as an input a feature of a detected object and determining a regression loss associated with parameters based on the feature of the detected object (i.e., Choi discloses object detection network 300 including three layers 322, one of which “refines the RoI [region of interest] location with a bounding box regressor”; Figure 3; paragraph [0051]);
determining a classification loss for a classification score for the feature of the detected object (i.e., layers 322 include two layers for subcategory classification and object class classification; paragraphs [0051]-[0052]); and
generating a multi-task loss based on the regression loss and the classification loss (i.e., “a multi-task loss for joint object class classification, subcategory classification, and bounding box regression”; paragraph [0053]).
Regarding claim 23, Choi discloses that determining the regression loss comprises determining the regression loss using the parameters of a bounding box of the feature (i.e., bounding box regression, paragraphs [0052]-[0053])..
Regarding claim 24, Choi discloses that the regression loss and the classification loss are determined concurrently (i.e., layers 322 operate in parallel as shown in Figure 3)..
Regarding claim 25, Choi discloses that the parameters are obtained from hierarchical convolutional features at least in the sense that Choi discloses that the system moves hierarchically from coarser object category classification (e.g., “car, pedestrian and cyclist”; paragraph [0060]) to finer subcategory classification (paragraphs [0018], [0038], and [0060]-[0066]).
Regarding independent claim 26, Choi discloses a method (Figures 1-3) comprising:
determining a regression loss associated with parameters based on a feature of a detected object in an image (i.e., Choi discloses object detection network 300 including three layers 322, one of which “refines the RoI [region of interest] location with a bounding box regressor”; Figure 3; paragraph [0051]);
determining a classification loss for a classification score for the feature of the detected object (i.e., layers 322 include two layers for subcategory classification and object class classification; paragraphs [0051]-[0052]); and
generating a multi-task loss based on the regression loss and the classification loss (i.e., “a multi-task loss for joint object class classification, subcategory classification, and bounding box regression”; paragraph [0053]).
Regarding claim 28, Choi discloses that determining the regression loss comprises determining the regression loss using the parameters of a bounding box of the feature (i.e., bounding box regression, paragraphs [0052]-[0053]).
Regarding claim 29, Choi discloses that the regression loss and the classification loss are determined concurrently (i.e., layers 322 operate in parallel as shown in Figure 3).
Regarding claim 30, Choi discloses that the parameters are obtained from hierarchical convolutional features at least in the sense that Choi discloses that the system moves hierarchically from coarser object category classification (e.g., “car, pedestrian and cyclist”; paragraph [0060]) to finer subcategory classification (paragraphs [0018], [0038], and [0060]-[0066]).
Regarding independent claim 31, Choi discloses an apparatus (Figures 1-3) comprising:
a regression loss circuit configured to determine a regression loss associated with parameters based on a feature of a detected object in an image object (i.e., Choi discloses object detection network 300 including three layers 322, one of which “refines the RoI [region of interest] location with a bounding box regressor”; Figure 3; paragraph [0051]);
a classification loss circuit configured to determine a classification loss for a classification score for the feature of the detected object (i.e., layers 322 include two layers for subcategory classification and object class classification; paragraphs [0051]-[0052]); and
a loss generator circuit configured to generate a multi-task loss based on the regression loss and the classification loss (i.e., “a multi-task loss for joint object class classification, subcategory classification, and bounding box regression”; paragraph [0053]).
Regarding independent claim 32, Choi discloses a system (Figures 1-3) comprising:
an input circuit configured to receive an input image having an object (e.g., input image 110; Figure 1; paragraph [0023]);
a convolutional neural network circuit configured to receive the input image from the input circuit and to generate hierarchical convolutional features (i.e., networks including region proposal network 120 and object detection network 130; paragraph [0023]); and
an object detection classification circuit 130 comprising:
a regression loss circuit configured to determine a regression loss associated with parameters based on a feature of a detected object in the input image (i.e., Choi discloses object detection network 300 including three layers 322, one of which “refines the RoI [region of interest] location with a bounding box regressor”; Figure 3; paragraph [0051]);
a classification loss circuit configured to determine a classification loss for a classification score for the feature of the detected object (i.e., layers 322 include two layers for subcategory classification and object class classification; paragraphs [0051]-[0052]); and
a loss generator circuit configured to generate a multi-task loss based on the regression loss and the classification loss “a multi-task loss for joint object class classification, subcategory classification, and bounding box regression”; paragraph [0053]),
wherein the parameters are obtained from the hierarchical convolutional features (i.e., Choi discloses that the parameters are obtained from hierarchical convolutional features at least in the sense that Choi discloses that the system moves hierarchically from coarser object category classification, e.g., “car, pedestrian and cyclist” to finer subcategory classification; paragraphs [0018], [0038], and [0060]-[0066]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 22 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Wolf.
Regarding claims 22 and 27, Choi discloses an apparatus and method as discussed above with regard to claims 21 and 26 respectively, including determining a regression loss associated with parameters based on a feature of a detected object but does not specifically disclose that determining the regression loss comprises determining the regression loss using the parameters of an alignment of the feature.
However, Wolf teaches an apparatus and method that is related to the one disclosed by Choi, including detecting an object in an image and determining a classification of the object (Wolf, Abstract; paragraph [0019]), and further teaches determining a loss using parameters of an alignment of the feature (Wolf, paragraphs [0046]-[0053]). Regarding claims 22 and 27, it would have been obvious to a person of ordinary skill in the art to use parameters of an alignment of the feature as taught by Wolf in the apparatus and method disclosed by Choi in order to advantageously better detect and identify objects in various orientations and angles in the image.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 11 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. US 10,032,067 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because reissue claim 11 recites essentially a subset of elements and limitations recited in claim 1 of U.S. Patent No. 10,032,067 B2, as shown in the table below:
Reissue claim 11
Claim 1 of US 10,032,067 B2
(limitations not in reissue claim 11 are marked with strikethrough)
11. A system to detect objects in an input image, the system comprising:
1. A system to recognize objects in an image, the system comprising:
an alignment network that is configured to be executed by at least one processor, wherein the alignment network receives as an input a first feature for a detected object in the input image, the alignment network to determine an alignment regression loss associated with alignment parameters based on the first feature for the detected object;
a face alignment regression network to determine a regression loss for alignment parameters based on the first hierarchical-calculated feature for the detected object,
a detection box network that is configured to be executed by the at least one processor, wherein the detection box network receives as an input the first feature for the detected object, the detection box network to determine a bounding-box regression loss for a bounding box for the first feature for the detected object; and
a detection box regression network to determine a regression loss for detected boxes based on the first hierarchical-calculated feature for the detected object, each detected box being a region of interest in the input image,
a loss generator that is configured to be executed by the at least one processor, wherein the loss generator receives an output from the alignment network of the alignment regression loss and an output from the detection box network of the bounding-box regression loss and generates a multi-task loss function based on the alignment regression loss and the bounding-box regression loss.
wherein the object detection network further comprises: a weighted loss generator to generate a multi-task loss function comprising
The limitations of reissue claim 11 are essentially found in claim 1 of US 10,032,067 B2. Given claim 1 of US 10,032,067 B2, it would have been obvious to a person of ordinary skill in the art to create reissue claim 11 by simply removing and slightly rewording limitations.
Allowable Subject Matter
Claims 1-20 may contain allowable subject matter if Applicant overcomes the rejections under 35 U.S.C. 251 and double patenting as discussed above.
The prior art does not specifically disclose or fairly suggest a system to detect objects in an input image including the combination of all of the elements, steps, and limitations recited in claims 1-20 (including all of the limitations of any respective parent claims), particularly including:
a loss generator that is configured to be executed by the at least one processor, wherein the loss generator receives an output from the alignment network of the alignment regression loss and an output from the first classification network of the classification loss and generates a multi-task loss function based on the alignment regression loss and the classification loss (e.g., claim 1); or
a loss generator that is configured to be executed by the at least one processor, wherein the loss generator receives an output from the alignment network of the alignment regression loss and an output from the detection box network of the bounding-box regression loss and generates a multi-task loss function based on the alignment regression loss and the bounding-box regression loss (e.g., claim 11).
Conclusion
Applicant is reminded of the continuing obligation under 37 CFR 1.178(b), to timely apprise the Office of any prior or concurrent proceeding in which this reissue application is or was involved. These proceedings would include interferences, reissues, reexaminations, and litigation. Applicant is further reminded of the continuing obligation under 37 CFR 1.56, to timely apprise the Office of any information which is material to patentability of the claims under consideration in this reissue application. These obligations rest with each individual associated with the filing and prosecution of this application for reissue. See also MPEP §§ 1404, 1442.01 and 1442.04.
Applicant is notified that any subsequent amendment to the specification and/or claims must comply with 37 CFR 1.173(b).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/patents/laws/interview-practice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Any inquiry concerning this communication or earlier communications from the examiner, or as to the status of this proceeding, should be directed to Examiner Christina Leung at telephone number (571) 272-3023; the Examiner’s supervisor, SPE Patricia Engle at (571) 272-6660; or the Central Reexamination Unit at (571) 272-7705.
/CHRISTINA Y. LEUNG/Primary Examiner, Art Unit 3991
Conferees:
/DEANDRA M HUGHES/Reexamination Specialist, Art Unit 3992
/Patricia L Engle/SPRS, Art Unit 3991
Prosecution Insights