Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is:
“…implementing a plurality of steps of tracking homologous points…” in claim 8.
Because this claim limitation is being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it is being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this limitation interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation recites sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "images SC” in line 9 and 11-12. There is insufficient antecedent basis for this limitation in the claim. There is no previous mention of this subset. There is only mention of subset SC. It is unclear if this is meant to be a new subset of images, or if this is a typo meant to indicate the subset is that of subset SC.
The term “SC” will be read as “SC” for each instance of the term throughout claims 1-12. This interpretation is further supported by the specification, in which there is no mention of a subset “SC”, instead each instance is Sc, including in portions that mirror the claim language (like on page 2 lines 15-30).
Claim 2 recites the limitations “…the convex envelope…” and "…the centres P…” in line 4. There is insufficient antecedent basis for this limitation in the claim. There is no previous mention of a convex envelope, nor mention of these center positions. As such, it is unclear what these limitations are meant to be referring to.
The limitation “…the convex envelope…” will be read as “…a convex envelope…” Additionally, "…the centres P…” will be read as "…centres P…”
Claim 5 recites the limitation "the geometric link Hi→ref between the image ItC and the reference image…” in line 7. There is insufficient antecedent basis for this limitation in the claim. There is no previous mention of this particular geometric link H. There is only the inclusion of a geometric link that connects the reference image and the images of the subset. However, there is no indication that this applies to every image of the subset. As there is no prior mention of this particular geometric link, it is unclear what is being referred back to in this limitation.
Claim 5 line 7 "the geometric link Hi→ref between the image ItC and the reference image…” will be read as "a geometric link Hi→ref between the image ItC and the reference image…”
Claim 5 recites the limitation "…the mosaic image Imos…” in line 2. There is insufficient antecedent basis for this limitation in the claim. There is no previous mention of this image in the claims. The only mosaic image mentioned is from claim 1 line 2, but this is not denoted as Imos. As there is no prior mention of this image, it is unclear if this is meant to be a new mosaic image being built, or if this is the mosaic image introduced in claim 1. Additionally, claims 6 and 7 are rejected due to their dependence and recitation of the term “the mosaic image Imos”.
Claim 1 line 2 "…a mosaic image…” will be read as "…the mosaic image (Imos)…” This is due to the language of claim 5 suggesting that it correlates to the step of building the mosaic image described in claim 1. This further preserves a definition of “Imos” for the operation described in claim 6 and claim 7.
Claim 11 recites the limitation "images SC” in line 11 and 13-14. There is insufficient antecedent basis for this limitation in the claim. There is no previous mention of this subset. There is only mention of subset SC. It is unclear if this is meant to be a new subset of images, or if this is a typo meant to indicate the subset is that of subset SC.
The term “SC” will be read as “SC” for each instance of the term throughout claim 11.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 13 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 13 indicates dependency to claim 13. A claim cannot claim dependence on itself as this would mean it references back and incorporates itself. Applicant may cancel the claim, amend the claim to place the claim in proper dependent form, rewrite the claim in independent form, or present a sufficient showing that the dependent claim complies with the statutory requirements.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3, 8, 10-13 are rejected under 35 U.S.C. 102 (a)(1)/(a)(2) as being anticipated by Liu et al. (applicant provided non-patent literature titled “Global and Local Panoramic Views for Gastroscopy: An Assisted Method of Gastroscopic Lesion Surveillance”; hereinafter “Liu”).
In re to claim 1, Liu teaches wherein: A method comprising: building a mosaic image from a sequence S of images of a hollow organ (section I subsection D para. 1 discloses the construction of panoramic images (understood as the mosaic image) from a plurality of images in a sequence. This is further shown to be the result of a gastroscope that images internal hollow organs, such as is exemplified in Figs. 4 and 9), said sequence S of images coming from an endoscope inserted into the hollow organ (section II para. 3 further suggests that imaging device used in the mentioned gastroscopic procedures is an endoscope. Additionally, section III subsection A para. 2 discloses the capture of image data using an endoscope), the building being implemented by an electronic device comprising a processor and a memory (section V para. 2 discloses that the system used comprises a CPU leveraging an i5 processor, and is thus understood to comprise a processor with memory) and comprising: selecting images, from said sequence S of images, delivering a subset of images Sc and a reference image Icref among this subset of images SC (section IV subsection B para. 3 discloses the selection of a frame (Ikover) from a frame set. It is understood that this frame is the reference frame. Additionally, it is understood that the set of images Ikover is selected from (Ipara ) is the subset of images);
determining geometric links H between the reference image I--cref previously determined and the images of the subset of images SC (section IV subsection B para. 3 discloses the matching of Ikover to frames in the subset of images (correspondent to the claims) based on the plane correspondence of the frames. This corresponding plane is understood as a geometric link between the frames, as it links the matching frames according to geometric concepts); and
building the mosaic image from the reference image I--cref, at least some of the images of the subset of images SC and the geometric links H of these images of the subset of images SC (section IV subsection B para. 4 discloses that the matching process is used to stitch image frames, thereby disclosing the building of the mosaic image using the reference image, subset images, and geometric links H, each correspondent to the claims, respectively).
In re to claim 3 [dependent on claim 1], Liu teaches wherein: determining a geometric link with the reference image I--cref- comprises (individual plane selection; section IV subsection B para. 3 discloses the matching of Ikover to frames in the subset of images (correspondent to the claims) based on the plane correspondence of the frames. This corresponding plane is understood as a geometric link between the frames, as it links the matching frames according to geometric concepts. An individual plane selection is understood as the geometric link), for a current image Ii belonging to the subset of images Sc (section IV subsection B para. 2-3 discloses the selection of overlay images based on a percentage of overlay, with said image being denoted as I1para, understood as a current image), at least one step of calculating a homography Hi→ref between the reference image and the current image, said homography Hi→re being determined according to a rate of superposition τi,ref between the pixels of the reference image I--cref - and the pixels of the current image Ii (section IV subsection B para. 2-3 discloses an overlay determination that considers a degree of overlay between images. This is understood as the determination of homography according to a rate of superposition (being a percentage of overlay with respect to the reference image)).
As to claim 10, it is the device used to execute the method of claim 1. As such, it recites similar limitations to claim 1, and is rejected for the same reasons as provided above.
As to claim 11, it is the non-transitory computer readable medium used to execute the method of claim 1. As such, it recites similar limitations to claim 1, and is rejected for the same reasons as provided above.
In re to claim 12 [dependent on claim 1], Liu teaches wherein: comprising receiving the sequence S of images from the endoscope (section II para. 3 further suggests that imaging device used in the mentioned gastroscopic procedures is an endoscope. Additionally, section III subsection A para. 2 discloses the capture of image data using an endoscope)that is inserted into the hollow organ (section I subsection D para. 1 discloses the construction of panoramic images (understood as the mosaic image) from a plurality of images in a sequence. This is further shown to be the result of a gastroscope that images internal hollow organs, This is further shown to be the result of a gastroscope that images internal hollow organs, such as is exemplified in Figs. 4 and 9.
As to claim 13, by virtue of containing substantially the same claim limitations as claim 12, it is rejected for the same reasons as provided for claim 12.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Turan et al. (applicant provided non-patent literature titled “Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots”; hereinafter “Turan”) and Kubo (US publication 20220414885 A1; hereinafter Kubo).
In re to claim 8 [dependent on claim 1], Liu does not explicitly teach wherein: the sequence S of images of the hollow organ is obtained by implementing a plurality of steps of tracking homologous points of images acquired by an operator of the endoscope during image captures carried out inside the hollow organ.
However, in the same field of endeavor, Turan teaches wherein: the sequence S of images of the hollow organ is obtained by implementing a plurality of steps of tracking homologous points (section 3.3 para. 1 discloses the matching of feature between images. Further, this stitching takes into consideration motion based alignment according to section 3.3 para. 2, this is understood as the tracking of homologous points, with each bullet point of presented in section 3.3 para. 1-2 being a plurality of steps done to perform this tracking).
Turan, like Liu, discloses an endoscope enabled imaging system that constructs a mosaic image.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective
filing date of the claimed invention to modify Liu, to include tracking of homologous points, as taught by Turan. The motivation for the proposed modification would have been to allow for adjustments to images with consideration of 2D motion estimations, as is performed in Turan section 3.3 para. 2 with regard to image alignment for further image stitching.
Liu, in view of Turan, does not explicitly teach wherein: that images are acquired by an operator of the endoscope during image captures carried out inside the hollow organ.
However, in the same field of endeavor, Kubo teaches wherein: that images are acquired by an operator of the endoscope during image captures carried out inside the hollow organ ([0072] discloses image capture being performed by the user. Additionally, [0110] discloses the use of the endoscope to image an area within an organ).
Kubo, like Liu, discloses a system that captures image data using an endoscope in order to process said image data in additional operations.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective
filing date of the claimed invention to modify Liu, in view of Turan, to have operator captured image data, as taught by Kubo. The motivation for the proposed modification would have been to better enable user agency by allowing them to select what image they want to acquire.
Allowable Subject Matter
Claims 2, 4, 5-7, and 9 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is an examiner’s statement of reasons for allowance for claim 2. The claimed
features of claim 2 are not anticipated nor obvious in view of prior art of record.
The closest known prior art Liu teaches mosaic image generation using images captured from an endoscope. It further discloses that a sequence of images is used to produce a reference image. Additionally, as shown in Fig. 4, there is a collection of images which includes a convex envelope of images selected to form the mosaic image. However, there is no explicit disclosure that a plurality of centers of images is used to determine the a subset. Liu describes the use of image points in section IV subsection C para. 2 for the stitching of faces, but these are not suggested to be center points. It additionally does not suggest use of the subset to select the reference image in order to maximize a distance to a center of the set of images.
Trinh et al. (applicant provided non-patent literature titled “MOSAICING OF IMAGES WITH FEW TEXTURES AND STRONG ILLUMINATION CHANGES:APPLICATION TO GASTROSCOPIC SCENES”; hereinafter “Trinh”) discloses a mosaic image construction using endoscope imaging. It further shows the generation of the mosaic from a plurality of image (shown in Fig. 5). Additionally, there is consideration of center positions according to section 2.2 para. 2 and section 3 para. 2. However, it does not indicate use of a subset to select the reference image in order to maximize a distance to a center of the set of images.
Niaf et al. (US publication 20200082149 A1; hereinafter “Niaf”) discloses a system that generates a composite image from a plurality of image portions. It additionally discloses the consideration of center points according to [0050]. However, it does not utilize endoscopic data, nor does it further disclose the use of a subset to select the reference image in order to maximize a distance to a center of the set of images.
Additionally, the other known prior art or record do not address all the limitations of the
independent claims without the use of impermissible hindsight bias. As such, claim 2’s limitations are
neither anticipated nor rendered obvious in view of prior art of record.
The following is an examiner’s statement of reasons for allowance for claim 4. The claimed
features of claim 4 are not anticipated nor obvious in view of prior art of record.
Liu teaches mosaic image generation using images captured from an endoscope. It further discloses the consideration of image overlap, as shown in Figs. 4 and 8. Consideration of the geometric relationships between images is also undertaken between a reference image and a given image in a subset, as is discussed in section IV subsection B para. 3. However, there is no explicit mention that there is a homography calculation for the sake of the geometric link that is performed according to a rate of superposition.
Turan discloses the stitching of images with respect to endoscopic data. It further performs homography determinations that consider the degree of overlap between a reference image and another image (serving as consideration of image superposition). However, there is no explicit indication that there is a consideration of two intermediate homographies with respect to an intermediate image.
Sinharoy et al. (US publication 20220070426 A1; hereinafter “Sinharoy”) discloses a system for the restoration of FOV images. It further discusses the determination of homography between images per [0096]. This homography determination additionally considers the superposition between images (specifically regarding a reference image). However, while a consideration is made for the superposition between a reference image and another image, it does not indicate a consideration of some form of intermediate homographies.
Additionally, the other known prior art or record do not address all the limitations of the
independent claims without the use of impermissible hindsight bias. As such, claim 4’s limitations are
neither anticipated nor rendered obvious in view of prior art of record.
The following is an examiner’s statement of reasons for allowance for claim 5. The claimed
features of claim 5 are not anticipated nor obvious in view of prior art of record.
Liu teaches mosaic image generation using images captured from an endoscope. Additionally, it indicates the selection of an image from a subset in order to perform further operations with respect to a reference image. However, it does not explicitly consider if an image is the farthest image from the reference image.
Turan discloses the stitching of images with respect to endoscopic data to generate a reconstructed image. It additionally performs affine transformations of an image with respect to a reference image, as described in section 3.3 para. 2. However, it does not indicate that the transformation is a part of an iterative process that both excludes pixels from a mosaic image to generate a combined image, and adds pixels from said combined image into the mosaic image while further deleting the image that was transformed from a subset.
Niaf discloses a system that generates a composite image from a plurality of image portions. However, it also does not perform image transformations while to further exclude and add pixels in the manner required by the limitations of claim 5.
Additionally, the other known prior art or record do not address all the limitations of the
independent claims without the use of impermissible hindsight bias. As such, claim 4’s limitations are
neither anticipated nor rendered obvious in view of prior art of record.
The following is an examiner’s statement of reasons for allowance for claim 9. The claimed
features of claim 9 are not anticipated nor obvious in view of prior art of record.
Liu teaches mosaic image generation using images captured from an endoscope. It further discloses use of a display, as shown in Fig. 5. However, it does not explicitly disclose that, before selecting images to generate a subset to select an initial image from a sequence of images. While the system performs image selection from a subset (as described in section IV subsection B para. 3) and further generates an additional subset and additional image selection (see section IV subsection C), it does not additionally indicate the display of the image sequence on the screen as a part of image selection. Nor does it indicate the calculation and display of some approximate mosaic image separate from the mosaic image described in independent claim 1.
Turan discloses the stitching of images with respect to endoscopic data to generate a reconstructed image. However, it also does not disclose the selection of images prior to generation of a subset (as would be required by the claim limitations of claim 9 lines 1-3) to further include display of all the required image data of claim 5. There is no explicit mention within Turan of the use of a display in its processes of image selection.
Trinh discloses a mosaic image construction using endoscope imaging. It further teaches the selection of reference image data from a sequence and considers the coordinates of said image, as per section 3 para. 1. It further indicates that the system selects images in a consecutive manner. However, it does not explicitly disclose that the images are displayed when selecting the initial image, nor does it indicate the calculation and display of an approximate mosaic image is performed. Further, it does not explicitly disclose the process of selecting a reference image is preceded by a selection process shown on a visualisation screen, as would be required according to the limitation of claim 9 lines 1-3.
Kubo teaches the use of a display of image data resultant from the endoscope. However, it also does not explicitly disclose the process of selecting a reference image being preceded by a selection process shown on a visualisation screen, as would be required according to the limitation of claim 9 lines 1-3.
Additionally, the other known prior art or record do not address all the limitations of the
independent claims without the use of impermissible hindsight bias. As such, claim 4’s limitations are
neither anticipated nor rendered obvious in view of prior art of record.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN M COOMBER whose telephone number is (571)270-0950. The examiner can normally be reached Monday - Friday 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN M COOMBER/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights