DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because according to MPEP 2111.01, the broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine-readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C § 101 as covering non-statutory subject matter.
In this case, claims 1-20 cover both transitory subject matter and non-transitory subject matter (see paragraphs 0042, 0044). Thus, claims 1-20 are rejected under 35 U.S.C 101. The USPTO suggests these claims may be amended to narrow the claim to cover only statutory embodiments to avoid rejections under 101 by adding the limitation “non-transitory" to the claims 1-20.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 12-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Independent claim 12 recites the limitation "the colonoscope user-interface”, “the colonoscope" in lines 9-10,13. There is insufficient antecedent basis for these limitations in the claims.
It is also noted that the claim term “adapted for” could be interpreted as language that suggests or makes optional but does not require the function to be performed or does not limit the scope of the claim limitation (see MPEP 2111.04).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 8-12, 14, 16-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zur (US 20200387706).
Regarding claim 1, Zur discloses at least one machine-accessible storage medium (computer readable storage medium), that provides instructions that, when executed by a machine (processor - see paragraphs 0066-0067, figure 2), will cause the machine to perform operations comprising:
generating a colonoscopy user-interface for display on a screen (generating a colonoscopy user-interface (GUI) for display on a screen – see include, but are not limited to, figures 1, 4-7, 13-17, paragraphs 0003-0004, 0009-0010, 0047),
outputting a live video feed received from a colonoscope for display within a video region of the colonoscopy user-interface (outputting a real time video images received from a video camera at the end of the colonoscope for display within a video region of the GUI –see include, but are not limited to, figures 1, 14-17, paragraphs 0002-0004, 0047, 0056, 0061) ;
annotating a polyp on the live video feed within the video region to visually indicate a location of the polyp with an annotation of the polyp (identifying or marking or generating instructions, augmenting for depicting a polyp on current image of real time video/images within the video region to visually indicate a location of the polyp with an indication/visual marking such as a box or region of interest (ROI) for the polyp – see include, but are not limited to, figures 1, 16, paragraphs 0047, 0125, 0127, 0208, 0212, 0221-0223);
tracking a directional movement of the colonoscope after annotating the polyp (tracking direction and/or orientation of the colonoscope camera after marking/identifying/augmenting the polyp – see include, but are not limited to, figures 1, 8, paragraphs 0048, 0056, 0092, 0108, 0222-0223) ; and
after the polyp is no longer visible on the live video feed within the video region, generating a lost polyp navigation aid for display within the colonoscopy user-interface to guide the colonoscope back to the polyp, wherein the lost polyp navigation aid indicates a guide direction to retrace the directional movement of the colonoscope based upon tracking the directional movement (after the polyp/ROI is located externally to the image (for example, no longer depicted in image 1910, or previous polyp which has disappeared from the current captured image) on the real time video within the video region, generating an polyp navigation aid/arrow 608/1912 for display within the GUI to guide the colonoscope back to the polyp/ROI, wherein the polyp navigation aid/arrow 608/1912 indicates a guide direction to retrace the directional movement of the colonoscope and point to the direction of the polyp 604/1904 – see include, but are not limited to, figures 6, 16, paragraphs 0031, 0049, 0054, 0061, 0081, 0115, 0131, 0208, 210-211, 222-223).
Regarding claim 2, Zur discloses the at least one machine-accessible storage medium of claim 1, wherein the live video feed and the lost polyp navigational aid are both contemporaneously presented within the colonoscopy user-interface (real time video captured by the colonoscope and arrow (608/1912) pointing to the direction of the disappeared polyp are both displayed on the GUI – see include, but are not limited to, figures 6, 16, paragraphs 0115, 0131, 0208, 210-211, 222-223) .
Regarding claim 3, Zur discloses the at least one machine-accessible storage medium of claim 1,wherein the annotation of the polyp is automatic when the polyp is displayed on the live video feed (identifying/marking of the poly is automatic when the polyp is displayed on the real time video image- see include, but are not limited to, figures 1, 4, 6, 15-17, paragraphs 0009, 0044, 0098, 0100, 0115, 0162, 0187, 0213, 0220, 0222-0223) .
Regarding claim 4, Zur discloses the at least one machine-accessible storage medium of claim wherein the annotation of the polyp is removed from the video region when the polyp is no longer displayed on the live video feed (e.g., marking/augmenting/box of the polyp/ROI 604/1094 is removed/disappeared from the video region when the polyp 604/1904 is no longer depicted/disappeared on the real time image/video or external to the real time images– see include, but are not limited to, figures 6, 16, paragraphs 0050, 0061, 0064, 0110, 0131, 0222).
Regarding claim 8, Zur discloses the at least one machine-accessible storage medium of claim 1, further providing instructions that, when executed by the machine, will cause the machine to perform further operations, comprising:
detecting return of the polyp on the live video feed (detecting a return/reappearing/recapturing of the polyp (e.g., 1904 in the last two images) on the real time video image – see include, but are not limited to, figures 16-17, paragraphs 0210-0211, 0221) ; and
reannotating the polyp on the live video feed within the video region to visually indicate the location of the polyp (re-marking or re-depicting the polyp/ROI on the real time video image within the video region to visually indicate the location of the polyp 1904 – see include, but are not limited to, figures 6, 16-17, paragraphs 0131, 0208, 0211).
Regarding claim 9, Zur discloses the at least one machine-accessible storage medium of claim 8, further providing instructions that, when executed by the machine, will cause the machine to perform further operations, comprising:
capturing an image of the polyp contemporaneously with the annotation (capturing an image of the polyp 1904/ROI with the annotation/marking/box/icon, etc. – see include, but are not limited to, figures 6, 16-17, paragraphs 0131, 0208, 0211-0212, 0221-0223); and
outputting the captured image of the polyp for display within the colonoscopy user-interface after reannotating the polyp on the live video feed to permit visual comparison between the reannotated polyp and the captured image (outputting the captured image of the polyp/ROI for display within the GUI after recapturing/remarking, etc. the polyp/ROI on the real time image to permit visual comparison between the remarked/recaptured polyp/ROI and the captured image – see include, but are not limited to, figures 6, 16-17, paragraphs 0131, 0208, 0211-0212, 0221-0223) .
Regarding claim 10, Zur discloses the at least one machine-accessible storage medium of claim 1, wherein generating the lost polyp navigation aid further comprises generating directional indication arrows, directional text, positional text, or a combination thereof (e.g., generating directional indicated arrow 608/1912 – see include, but are not limited to, figures 6, 16, paragraphs 0049, 0054, 0061, 0092, 0115, 0131, 0210, 0211, 0222).
Regarding claim 11, Zur discloses the at least one machine-accessible storage medium of claim 1, wherein generating the lost polyp navigation aid is automatic when a tool is present in the live video feed and the polyp is no longer visible on the live video feed within the video region (e.g., the disappeared/no longer-displayed polyp navigation aid/arrow is automatic when a tool is present in real time image and the polyp is no longer visible or disappeared or external on the real time video image within the video region -- see include, but are not limited to, figures 1, 6, 16-17, paragraphs 0009, 0044, 0098, 0100, 0115, 0131, 0162, 0187, 0210-0213, 0220, 0222-0223) .
Regarding claim 12, the limitations that correspond to the limitations of claim 1 are analyzed as discussed in the rejection of claim 1. Particularly, Zur discloses at least one machine-accessible storage medium that provides instructions that, when executed by a machine, will cause the machine (computing device with processor – see figures 1-2 and discussion in claim 1) to output a signal for rendering a user-interface to a display (e.g., user interface with display 226 – see include, but are not limited to, figures 1-2, and discussion in the rejection of claim 1), the user-interface adapted for visualizing an endoscopy procedure (see figures 1-2, 6, 15-17 and paragraphs 0002-0003, 0009-0010 and discussion in claim 1), the user-interface comprising:
a video region in which a live video feed received from an endoscope is displayed (video region of the GUI in which the real time images/video received from an endoscope/colonoscopy is displayed – see discussed in the rejection of claim 1 and include, but are not limited to, figures 2, 4-7, 15-17);
an annotation of a polyp on the live video feed within the video region to visually indicate a presence of the polyp (an annotation/marking/ROI/box, etc. of polyp on the real time video within the video region/image to visually indicate a presence of the polyp (408/604-1904/2008/2010 – see discussion in the rejection of claim 1 and include, but are not limited to figures 1-2, 4-7, 15-17); and
a lost polyp navigation aid (e.g., arrow 608/1912) within the colonoscopy user-interface to guide the colonoscope back to the polyp after the polyp is no longer visible on the live video feed within the video region, wherein the lost polyp navigation aid indicates a guide direction to retrace a directional movement of the colonoscope occurring since the polyp was last visible on the live video feed within the video region (see discussion in the rejection of claim 1 and include, but are not limited to, figures 6, 16, paragraphs 0208-0211, 0221-0223).
Regarding claim 14, Zur discloses the least one machine-accessible storage medium of claim 12, wherein the user-interface further comprises. a polyp identifier graphic displayed within the colonoscopy user-interface and indicative of whether the polyp is currently visible on the live video feed within the video region (a polyp identifier graphic such as a box, icon, ROI, etc. displaying within the GUI and indicative of whether the polyp 604, 1904 is currently visible on the real time video feed within the video region – see include, but are not limited to, figures 1, 6, 15-17, paragraphs 0131, 0208-0212, 0221-0223).
Regarding claim 16, Zur discloses the least one machine-accessible storage medium of claim 12, wherein the user-interface further comprises: a reannotation of the polyp on the live video feed within the video region to visually indicate the presence of the reannotated polyp (see similar discussion in the rejection of claim 8, and see include, but are not limited to, figures 6, 16-17, paragraphs 0131, 0208, 0211).
Regarding claim 17, Zur discloses the least one machine-accessible storage medium of claim 16, wherein the user-interface further comprises a captured image of the polyp captured during annotating the polyp, the captured image displayed after the reannotation of the polyp on the live video feed to permit visual comparison between the reannotated polyp and the captured image (see similar discussion in the rejection of claim 9 and include, but are not limited to, figures 16-17).
Regarding claim 18, Zur discloses the least one machine-accessible storage medium of claim 12, wherein the user-interface further comprises. directional indication arrows, directional text, positional text, or a combination thereof displayed within the colonoscopy user-interface and indicative of a guide direction retracing the directional movement of the colonoscope after annotation of the polyp (see similar discussion in the rejection of claim 10 and figures 6, 16).
Regarding claim 19, Zur discloses the least one machine-accessible storage medium of claim 12, wherein the user-interface further comprises automatic display of the lost polyp navigation aid when a tool is present in the live video feed and the polyp is no longer visible on the live video feed within the video region (see similar discussion in the rejection of claim 11).
Regarding claim 20, Zur discloses the least one machine-accessible storage medium of claim 12, wherein the user-interface comprises a colonoscopy user-interface (see for example, figure 17), the endoscopy procedure comprises a colonoscopy procedure, the endoscope comprises a colonoscope, and the polyp comprises a polyp on a colon (see include, but are not limited to, figures 16-17, paragraphs 0001-0002, 0009, 0017).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-7, 13, 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zur (US 20200387706) as applied to claim 1 or claim 12 above and in view of Tachibana (US 20220000337).
Regarding claim 5, Zur discloses the at least one machine-accessible storage medium of claim 1, further providing instructions that, when executed by the machine, will cause the machine to perform further operations, comprising:
assigning an identification information to the polyp contemporaneously with the annotation (assigning in identification of a box, marking, indicator, etc. to the polyp 408A, 604/1904/2008, 2010 – see include, but are not limited to, figures 4, 6, 16-17 and discussion in the rejection of claim 1) ; and
outputting the polyp identification information for display within a procedure data region of the colonoscopy user-interface (see include, but are not limited to, figures 4, 6, 16-17 and discussion in the rejection of claim 1).
Zur does not explicitly disclose the polyp identification information comprises polyp identification number.
Tachibana discloses assigning an identification number (e.g., polyp ID with identification number 14, 15, 16) to polyp contemporaneously with annotation (see include, but are not limited to, figure 14A, paragraphs 0080, 0083-0084) ; and
outputting the polyp identification number for display within a procedure data region of the colonoscopy user-interface (see include, but are not limited to, figure 14a, paragraph 0083-0084).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the polyp identification information of Zur to comprises polyp identification number as taught by Tachibana in order to yield predictable result of allowing each polyp to be identified easily – see for example, paragraph 0084)
Regarding claim 6, Zur in view of Tachibana discloses the at least one machine-accessible storage medium of claim 5, wherein outputting the polyp identification number comprises outputting a polyp identifier graphic indicative of whether the polyp is visible on the live video feed within the video region (see include, but are not limited to, Zur: figures 6, 16-17, Tachibana: figure 14a, paragraph 0083-0084).
Regarding claim 7, Zur in view of Tachibana discloses the at least one machine-accessible storage medium of claim 5,further providing instructions that, when executed by the machine, will cause the machine to perform further operations, comprising: outputting a polyp count graphic indicative of the total number of polyps assigned an identification number to the colonoscopy user-interface (e.g., output or display a count or aggregated number of polyps - see include, but are not limited to, Zur: figure 17, Tachibana: figures 5, 12, 13-14b, 16, 17, 20, 25, paragraphs 0041, 0074-0075, 0083-0085).
Regarding claim 13, Zur in view of Tachibana discloses the least one machine-accessible storage medium of claim 12, wherein the user-interface further comprises an identification number of the polyp displayed within a procedure data region of the colonoscopy user-interface (see similar discussion in the rejection of claim 5 and Tachibana: figure 14a, paragraphs 0083-0084).
Regarding claim 15, Zur in view of Tachibana discloses the least one machine-accessible storage medium of claim 12, wherein the user-interface further comprises: a polyp count graphic displayed within the colonoscopy user-interface and indicative of the total number of polyps assigned an identification number (see similar discussion in the rejection of claim 7 and include, but are not limited to, Zur: figure 17, Tachibana: figures 5, 12, 13-14b, 16, 17, 20, 25, paragraphs 0041, 0074-0075, 0083-0085).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Evans et al. (US 20210398676) discloses machine learning algorithms for detecting medical conditions, related system and related methods including detecting and annotating polyps (see also figures 6-8).
Freedman et al. (US 20210280312) discloses detecting deficient coverage in gastroenterological procedures.
Kronman et al. (US 10292570) discloses system and method for guiding and tracking a region of interest using an endoscope.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AN SON P HUYNH whose telephone number is (571)272-7295. The examiner can normally be reached 9:00 am-6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANH TUAN T. NGUYEN can be reached at 571-272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AN SON P HUYNH/Primary Examiner, Art Unit 3795
October 3, 2026
