DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-7, 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Zur (Pub. No.: US 2020/0387706) in view of Asatsuma et al (Pub. No.: US 2021/0193727)
Regarding claims 1, 16, Zur discloses a system for generating a target image, comprising:
an endoscope having an image collection component (camera) [see 0045, 0047, 0092];
a computing device (204) communicatively connected to the image collection component of the
endoscope [see 0077];
comprising a non-transitory computer-readable medium with instructions stored thereon, which
when executed by a processor perform steps [see 0066, 0079] comprising:
receiving at least one input image (the captured images as relied on as the input image,
emphasis added) from the image collection component of the endoscope [see 0075, 0095, 0134-0135]
by disclosing the 3D reconstruction neural network may be trained using a training dataset of pairs of 2D
endoscopic images defining input images [see 0135];
providing the at least one input image as an input to a machine learning algorithm [see 0099,
0102] by disclosing the image is fed into a detection neural network [see 0099] and one or more
endoscopic images of a sequential sub-set of the endoscopic images are fed into the detection neural
network [see 0103];
generating a target image (the 3D reconstructed image is relied on as the target image,
emphasis added) from the at least one input image using the machine learning algorithm [see fig 8,
0134-0135];
providing the at least one input image and the target image to a display driver [see 0064, 0103]
by disclosing the transformed location is the location that is presented on the display with frame
number i±2 [see 0064];
a display device, communicatively connected to the computing device, and configured to display the at least one input image and the target image see 0064, 0080] by disclosing Computing device 204 is connected between imaging probe 212 and display 226 [see 0080-0083].
Zur doesn’t disclose a first display for one eye of a user and a second display for a second eye of the user to display the at least one input image on the first display and the target image on the second display and thereby generating an artificial stereoscopic view
Nonetheless, Asatsuma et al disclose a first display for one eye of a user and a second display for a second eye of the user to display the at least one input image on the first display and the target image on the second display and thereby generating an artificial stereoscopic view [see 0412]
Therefore, it is obvious to one skilled in the art at the time the invention was filed and would
have been motivated to combine Zur and Asatsuma et al by using a first display for one eye of a user and a second display for a second eye of the user to display the at least one input image on the first display and the target image on the second display and thereby generating an artificial stereoscopic view; to more accurately grasp the depth of biological tissue.
Regarding claims 2, 17, Zur discloses wherein the at least one input image comprises a sequence
of at least five frames of a video recorded by the image collection component [see 0135-0136, 0161] by
disclosing the 3D reconstruction process may be trained using a large dataset (e.g., at least 100,000
images of the colon from at least 100 different colonoscopy videos, or other smaller or larger values)
[see 0136].
Regarding claim 3, Zur discloses wherein the image collection component is a camera [see 0045,
0047, 0092].
Regarding claim 5, Zur discloses wherein the computing device is positioned in the display
device [see 0080] by disclosing computing device 204 may be installed for each colonoscopy workstation
(e.g., includes imaging probe 212 and/or display 226) [see 0080]
Regarding claim 6, Zur discloses wherein the computing device is positioned in the endoscope
[see 0080] by disclosing computing device 204 may be installed for each colonoscopy workstation (e.g.,
includes imaging probe 212 and/or display 226) [see 0080]
Regarding claims 7, 18, Zur discloses wherein the machine learning algorithm is selected from a
convolutional neural network [see 0051, 0147], a generative/adversarial neural network, or a U-Net.
Claim(s) 4 is rejected under 35 U.S.C. 103 as being unpatentable over Zur (Pub. No.: US
2020/0387706) in view of Asatsuma et al (Pub. No.: US 2021/0193727) as applied to claim 1 above and further in view of Abitbol (Pub. No.: US 2018/0160885).
Regarding claim 4, Zur discloses the endoscope further comprising a tube with the image
collection component positioned at a distal end of the tube [see 0002]
Zur and Asatsuma et al don't disclose the tube having an outer diameter of at most 10 mm
Nonetheless, Abitbol discloses the tube having an outer diameter of at most 10 mm [see 0107]
by disclosing carrier element 1604 may be elongate, such as in the form of a hollow cylinder (or tube)
having a diameter of approximately 1/16" (2.5 mm) [see 0107].
Therefore, it is obvious to one skilled in the art at the time the invention was filed and would
have been motivated to combine Zur, Abitbol and Asatsuma et al by the tube having an outer diameter of at most 10 mm; to have a diameter small enough to navigate easily.
Claim(s) 8-9, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zur (Pub. No.:
US 2020/0387706) in view of Asatsuma et al (Pub. No.: US 2021/0193727) in view of Linard et al (Pub. No.: US 2016/0253801).
Regarding claim 8, Zur and Asatsuma et al don't disclose buffering a sequence of input images to process with the machine learning algorithm.
Nonetheless, Linard et al disclose buffering a sequence of input images [see abstract, 0018-
0019, 0065].
Therefore, it is obvious to one skilled in the art at the time the invention was filed and would
have been motivated to combine Zur, Linard et al, Asatsuma et al by buffering a sequence of input images; because the buffer acts as a temporary storage tank, allowing the camera to keep capturing images even while the memory card is being written to; Without a buffer, the camera would have to wait for each image to
be fully written to the card before taking the next shot. The buffer eliminates this delay, letting you
shoot faster and more fluidly.
Regarding claim 9, Zur discloses wherein the sequence comprises at least five input images
[see 0135-0136] by disclosing the 3D reconstruction process may be trained using a large dataset (e.g.,
at least 100,000 images of the colon from at least 100 different colonoscopy videos, or other smaller or
larger values) [see 0136].
Claim(s) 21 is rejected under 35 U.S.C. 103 as being unpatentable over Zur (Pub. No.: US
2020/0387706) in view of Asatsuma et al (Pub. No.: US 2021/0193727) in view of Godard et al (Pub. No.: US 2019/0213481)
Regarding claim 21, Zur, Asatsuma et al don't disclose upsampling the at least one input image using bilinear interpolation or strided transpose convolution.
Nonetheless, Godard et al disclose upsampling the at least one input image using bilinear
interpolation or strided transpose convolution [see 0011, 0047].
Therefore, it is obvious to one skilled in the art at the time the invention was filed and would
have been motivated to combine Zur, Asatsuma et al and Godard et al by upsampling the at least one input image using bilinear interpolation or strided transpose convolution; Upsampling corrects imbalanced data, which can lead to better model performance in machine learning and data analysis; Upsampling can reduce distortion and provide a more detailed representation of audio signals, potentially improving sound quality; Upsampling allows digital filters to operate at higher frequencies, reducing the impact of aliasing and ringing in audio streams and Upsampling can lead to better conversion accuracy in DACs, reducing artifacts and improving overall sound quality.
Claim(s) 22 is rejected under 35 U.S.C. 103 as being unpatentable over Zur (Pub. No.: US
2020/0387706) in view of Asatsuma et al (Pub. No.: US 2021/0193727) in view of Zagaynov et al (Pub. No.: US 2021/0224969)
Regarding claim 22, Zur, Asatsuma et al don't disclose training the machine learning model with a loss function; and adjusting a parameter weight of the target image based on the loss function.
Nonetheless, Zagaynov et al disclose training the machine learning model with a loss function; and adjusting a parameter weight of the target image based on the loss function [see 0028].
Therefore, it is obvious to one skilled in the art at the time the invention was filed and would
have been motivated to combine Zur, Asatsuma et al and Zagaynov et al by training the machine learning model with a loss function; and adjusting a parameter weight of the target image based on the loss function; to optimize prediction accuracy.
Response to Arguments
Applicant’s arguments, see REM, filed 6/22/2026, with respect to the rejection(s) of claim(s) 1-3, 5-7, 16-18 under 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Asatsuma et al (Pub. No.: US 2021/0193727).
The 101 rejection of the previous office is moot.
Applicants submit that Zur does not describe, explicitly or inherently, a display device comprising a first display for one eye of a user and a second display for a second eye of the user, or that the least one input image is displayed on the first display and the target image on the second display, thereby generating an artificial stereoscopic view/image.
The examiner agrees and Asatsuma et al disclose a first display for one eye of a user and a second display for a second eye of the user to display the at least one input image on the first display and the target image on the second display and thereby generating an artificial stereoscopic view [see 0412].
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOEL F BRUTUS whose telephone number is (571)270-3847. The examiner can normally be reached Mon-Sat, 11:00 AM to 7:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached at 571-270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOEL F BRUTUS/Primary Examiner, Art Unit 3797
Prosecution Insights