Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This communication is a non-Final office action in merits. Claims 1-14, as originally filed, are presently pending and have been elected and considered below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/3/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. DE10 2023 209 950.0, filed on 10/11/2023.
Drawings
The drawings are objected to under 37 CFR 1.83(a) because they fail to show names and/or illustrations of functional blocks in drawings as described in the specification. Any structural detail that is essential for a proper understanding of the disclosed invention should be shown in the drawing. MPEP § 608.02(d). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6, 9, 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over US 2019/0295282 A1, Smolyanskiy et al. (hereinafter Smolyanskiy) in view of US 2019/0371024 A1, Wisely Babu et al. (hereinafter Wisely Babu).
As to claim 1, Smolyanskiy discloses a method for image analysis comprises:
providing a first image of an object from a first perspective (Fig 1A: 102A; Fig 4; par 0035, receiving a first image);
providing a second image of the object from a second perspective (Fig 1A: 102B; Fig 4; pars 0035, receiving a second image);
forming a first feature map in an encoding layer of a convolutional neural network from the first image (Fig 1A; pars 0039, 0042, compute a first feature map corresponding to the first image);
acquiring a feature in the first feature map (Figs 1A-1B; pars 0036, 0074, extracting a feature in the first feature map);
generating an epipolar information item regarding the acquired feature for the second perspective (pars 0042-0043, generating epipolar lines from the first feature map (left) to the second feature ma (right)).
Smolyanskiy does not expressly disclose introducing the epipolar information item into a decoding layer of the convolutional neural network; and obtaining a second analysis image from the second image by decoding second feature maps of the second image comprising the epipolar information item by the decoding layer.
Wisely Babu, in the same or similar field of endeavor, further teaches generating an epipolar information item regarding the acquired feature for the second perspective (pars 0047, 0051, 0059); introducing the epipolar information item into a decoding layer of the convolutional neural network (Figs 1, 4, epipolar constrained layer being integrated into decoding block of the convolutional neural network; pars 0015); and
obtaining a second analysis image from the second image by decoding second feature maps of the second image comprising the epipolar information item by the decoding layer (Figs 4-5; pars 0032, 0039, 0045, 0048, 0050-0053, the second image being constructed ith epipolar constraint including epipolar lines being provided in an decoder of a CNN).
Therefore, consider Smolyanskiy and Wisely Babu’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Wisely Babu’s teachings in Smolyanskiy’s method to inject geometric constraints from multiple camera views into the feature reconstruction process.
As to claim 2, Smolyanskiy as modified discloses the method of claim 1, wherein the image analysis comprises a segmentation, a regression, and/or a detection (Smolyanskiy: pars 0003, 0005, 0103-0104, 0121, object detection).
As to claim 3, Smolyanskiy discloses the method of claim 1, wherein a differentiable operator is used in the introducing of the epipolar information item (Smolyanskiy: Figs 3A, 4; pars 0042-0043, 0055, 0057, 0075, 0077, 0080, claim 9, gradient decent based training of the image (differential operator) in encoder/decoder of the convolutional neural network; Wisely Babu: Fig 4; par 0056).
As to claim 4, Smolyanskiy discloses the method of claim 1, wherein the epipolar information item comprises a line that results from a first ray from the feature to a ray source in the first perspective and a ray geometry of the second perspective (Smolyanskiy: 3A, 4; pars 0042-0043, epipolar line from the feature map extended to the second feature map; Wisely Babu: pars 0049-0053).
As to claim 5, Smolyanskiy discloses the method of claim 4, wherein the line corresponds to a section line that results from an epipolar plane extending through the ray source in the second perspective parallel to the first ray, intersected by a second image plane in the second perspective (Smolyanskiy: pars 0042-0043; Wisely Babu: pars 0049-0053).
As to claim 6, Smolyanskiy discloses the method of claim 1, wherein the epipolar information item is generated in a separate feature map fed to another feature map in the decoding layer when introduced into the decoding layer (Smolyanskiy: Fig 4; pars 0043-0043; Wisely Babu: Figs 4-5; pars 0049-0053).
As to claim 9, Smolyanskiy as modified discloses the method of claim 1, wherein a respective epipolar information item is taken into account for each scaling level of a plurality of scaling levels of the decoding layer (Smolyanskiy: pars 0037, 0040, 0042-0044, down-sampling; Wisely Babu: pars 0047, 0057, up-sampling layers in decoder).
As to claim 13, it recites essentially functions and features recited in claims 1 and 3. Rejection of claims 1 and 3 are incorporated herein.
As to claim 14, it is an apparatus claim encompassed claim 1. Rejection of claim 1 is therefore incorporated herein.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Smolyanskiy in view of Wisely Babu and further in view of US 2023/0222654 A1, Fan et al. (hereinafter Fan).
As to claim 8, Smolyanskiy as modified discloses the method of claim 1, but does not expressly teach wherein the convolutional neural network has a U-Net architecture. Fan, in the same or similar field of endeavor, further teaches a convolutional neural network may possess U-Net structure (pars 0008, 0010). Therefore, consider Smolyanskiy as modified and Fan’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Fan’s teachings on U-Net structure in Smolyanskiy as modified’s convolutional neural network for accurate image segmentation and preservation of fine image details.
Claims 7, 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Smolyanskiy in view of Wisely Babu and further in view of US 2022/0028080 A1, Lavi et al. (hereinafter Lavi).
As to claim 7, Smolyanskiy as modified discloses the method of claim 1, but does not expressly teach wherein the first perspective is orthogonal to the second perspective. Lavi, in the same or similar field of endeavor, further teaches two perspectives (segment selections or profiles) may be orthogonal (pars 0210, 0249, 0370). Therefore, consider Smolyanskiy as modified and Lavi’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate Lavi’s teachings in Smolyanskiy as modified’s method for selecting orthogonal segments to capture complementary information as desired.
As to claim 10, Smolyanskiy as modified the method of claim 1, wherein the first image and the second image are each an X-ray image or a sonography image (Lavi: 0230, 0300, 0341, X-ray image).
As to claim 11, Smolyanskiy as modified the method of claim 10, wherein the first image and the second image are obtained with X-ray technology by way of a C-arm device or a CT scanner (Lavi: pars 0194, 0230, 0429, X-ray from CT scan).
Allowable Subject Matter
Claim 12 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Reasons for Allowance
Prior art of record (Smolyanskiy, Wisely Babu, Fan, and Lavi) neither discloses alone nor teaches in combination functions and features recited in claim 12.
Examiner’s Note
Examiner has cited particular column, line number, paragraphs and/or figure(s) in the reference(s) as applied to the claims for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the reference(s) in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUN SHEN whose telephone number is (571)270-7927. The examiner can normally be reached on Mon-Fri 8:30-5:50 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUN SHEN/
Primary Examiner, Art Unit 2662
Prosecution Insights