Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/29/2024, 03/07/2024, is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4-7, 11-12, 19, are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recites “wherein the controller is further configured to: dewarp the first subset of the source image data generating normalized image data; and warp or modify the normalized image data in the first subset generating the modified source image data simulating a second inclination angle.”
Here, as recited, first dewarping is done to generate the normalized image data and then wrapping is done to make modified image data. It is unclear why the dewarping is needed first if wrapping is done later. Appropriate correction/explanation is required.
Claim 19 encompasses similar defect as claim 4.
Claim 7 recites the limitation "wherein the controller is further configured to: generate a simulated mask enclosed about a perimeter of the first image, wherein the simulated mask includes an orientation cue identifying a direction of the rotation relative to the first image data demonstrating the first portion of the field of view simulating a second inclination angle. There is insufficient antecedent basis for the limitation “rotation” in the claim.
The claim also does not indicate regarding what is being rotated.
Claim 7, recites the limitation, “the first image,” in line 3.
Claim 11, recites the limitation, “the first image,” “the second image” in line 3.
There is insufficient antecedent base issue with this limitation which makes the claim unclear because it is unclear regarding which images these first and second images are referring to with respect to the images formed by the imaging system.
Claims 5-6, 12 are rejected for being dependent on a rejected base claim.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3 is/are rejected under 35 U.S.C. 102 as being anticipated by Shameli (US 20200201022).
Regarding claim 1, Shameli discloses a surgical imaging system (FIG. 1) comprising:
a scope comprising an optic element (FIG. 3A; camera 60 with a lens; para [0043]) aligned at a first inclination angle defining a native perspective;
an image sensor (camera 60; FIG. 3A) configured to capture source image data in a field of view transmitted through the optic element; and
a controller (control 56) configured to:
control the capture of source image data at the native perspective at the first inclination angle (Para [0054]);
select a first subset of the source image including a first portion of the field of view simulating a second inclination angle (Image is modified so that a visual effect similar to the rotation of the lens is produced. Para [0068]; A captured image for a field of view is used.); and
selectively output modified source image data simulating the second inclination angle to a display screen (Image is modified so that a visual effect similar to the rotation of the lens is produced. Para [0068]; This may be accomplished by applying image processing algorithms to modify and redisplay the image in real-time. Para [0068]; Experience of rotational orientation is provided. Para [0067]).
Regarding claim 2, Shameli discloses wherein the first subset is offset from a focal center of the field of view of the source image data based on a difference between the first inclination angle and the second inclination angle (Since algorithms brings the effect of rotation of the lens, Para [0069], this is same as having difference between the first inclination angle and the second inclination angle. Experience of rotational orientation is provided. Para [0067]).
Regarding claim 3, Shameli discloses wherein the controller is further configured to: detect one or more features in the source image data indicating a rotation of the scope relative to a camera body (The features in the image that correspond to the rotation be considered as the detected features because image signal corresponding to these features undergo the rotational effect. The claim does not define what the specific features in the image are.), wherein the selection of the first subset is responsive to the rotation of the scope (The image data that undergo rotational effect are considered as the subset of the data and since the data underdo rotational effect, the data are selected by the processor to undergo through the algorithm.).
Regarding claim 14, Shameli discloses wherein the generation of the first image data comprises applying an image correction algorithm, wherein the controller is further configured to: limit a range of the image correction algorithm to the first subset of the source image data for the first image data (Image is modified so that a visual effect similar to the rotation of the lens is produced. Para [0068]; A captured image for a field of view is used; This may be accomplished by applying image processing algorithms to modify and redisplay the image in real-time. Para [0068]; Data that is having a visual effect similar to the rotation of the lens is considered as the range of data.).
Regarding claim 16, Shameli discloses method (abstract; FIG. 1) for controlling a surgical imaging comprising:
controlling the capture of source image data (Imaging angle can be changed; FIG. 2; Para [0054]) in a field of view at a native perspective at the first inclination angle of a scope at a first inclination angle;
selecting a first subset of the source image including a first portion of the field of view simulating a second inclination angle (Image is modified so that a visual effect similar to the rotation of the lens is produced. Para [0068]; A captured image for a field of view is used.); and
selectively displaying modified source image data simulating the second inclination angle (Image is modified so that a visual effect similar to the rotation of the lens is produced. Para [0068]; This may be accomplished by applying image processing algorithms to modify and redisplay the image in real-time. Para [0068]; Experience of rotational orientation is provided. Para [0067]).
Regarding claim 17, Shameli discloses wherein the first subset is offset from a focal center of the field of view of the source image data based on a difference between the first inclination angle and the second inclination angle (Since algorithms brings the effect of rotation of the lens, Para [0069], this is same as having difference between the first inclination angle and the second inclination angle. Experience of rotational orientation is provided. Para [0067]).
Regarding claim 18, Shameli discloses detecting one or more features in the source image data indicating a rotation of the scope relative to a camera body (The features in the image that correspond to the rotation be considered as the detected features because image signal corresponding to these features undergo the rotational effect. The claim does not define what the specific features in the image are.), wherein the selection of the first subset is responsive to the rotation of the scope (The image data that undergo rotational effect are considered as the subset of the data and since the data underdo rotational effect, the data are selected by the processor to undergo through the algorithm.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 4-6, 19, is/are rejected under 35 U.S.C. 103 as being unpatentable over Shameli (US 20200201022) in view of Morales (US 20170230558).
Regarding claim 4, Shameli discloses warp or modify a normalized image data in the first subset generating the modified source image data simulating a second inclination angle (Since algorithms brings the effect of rotation of the lens, Para [0069], this includes modifying the normal image data taken by the camera to simulate the second inclination angle. Experience of rotational orientation is provided. Para [0067]).
Shameli does not expressly disclose wherein the controller is further configured to: dewarp the first subset of the source image data generating normalized image data.
Morales is directed to exposure control in an imaging system (abstract) and teaches wherein the controller is further configured to: dewarp the first subset of the source image data generating normalized image data (Distorted image due to lens angle of view is unwrapped to generate an unwrapped image. FIG. 1A; para [0005]; Unwarping the image data within the selected ROI to produce final unwarped image signals 32. para [0035]; Also see rejection under 112(b) above.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shameli to provide unwrapped image data before wrapping the image data in accordance with the teaching of Morales so that image data that correspond to the wider view angle could be normalized and presented to a viewer as needed.
Regarding claim 5, Shameli as modified teaches wherein the dewarping of the first subset is corrected for one or more distortions or magnifications of the native perspective at the inclination angle (Morales: distorted image due to lens angle of view is unwrapped to generate an unwrapped image.).
Regarding claim 6, Shameli discloses wherein the warping of the normalized image data is distorted (Since algorithms brings the effect of rotation of the lens, Para [0069], this includes modifying the normal image data taken by the camera to simulate the second inclination angle. Experience of rotational orientation is provided. Para [0067]), simulating a magnification or distortion at the second inclination angle (Algorithms brings the effect of rotation of the lens, Para [0069]).
Regarding claim 19, Shameli discloses warping or modifying the normalized image data in the first subset generating the modified source image data simulating a second inclination angle (Since algorithms brings the effect of rotation of the lens, Para [0069], this includes modifying the normal image data taken by the camera to simulate the second inclination angle. Experience of rotational orientation is provided. Para [0067]).
Shameli does not expressly disclose dewarping the first subset of the source image data generating normalized image data.
Morales is directed to exposure control in an imaging system (abstract) and teaches dewarping the first subset of the source image data generating normalized image data (Distorted image due to lens angle of view is unwrapped to generate an unwrapped image. FIG. 1A; para [0005]; Unwarping the image data within the selected ROI to produce final unwarped image signals 32.para [0035]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shameli to provide unwrapped image data before wrapping the image data in accordance with the teaching of Morales so that image data that correspond to the wider view angle could be normalized and presented to a viewer as needed.
Claim(s) 8, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shameli (US 20200201022) in view of Sasaki (US 20140204187).
Regarding claim 8, Shameli does not disclose wherein the controller is configured to generate the modified source image data by generating a virtual mask superimposed over a field stop mask of the scope about the field of view.
Sasaki is directed to an endoscope image processing device (abstract) and teaches wherein the controller is configured to generate the modified source image data by generating a virtual mask (Boundary region BR; L1 is an extraction boundary line for extracting the front-field-of-view image FR; FIG. 6A; para [0070], [0071]) superimposed over a field stop mask of the scope about the field of view (A side-field-of-view image SR; para [0070]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shameli to define a region in accordance with the teaching of Sasaki so that the defined region could be used for generating the modified image data. Further, the defining a region would help applying an algorithm for data modification.
Regarding claim 13, Shameli, as modified teaches wherein the one or more features comprise a field stop mask of the scope demonstrated in the source image data (Boundary region BR; L1 is an extraction boundary line for extracting the front-field-of-view image FR; FIG. 6A; para [0070], [0071]; The boundary region can be considered as a field stop mask because the boundary region can be used to define portion of area where the algorithm could be applied for providing tilting experience.).
Claim(s) 9, 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shameli (US 20200201022) in view of Morales (US 20170230558).
Regarding claim 9, Shameli does not expressly disclose wherein the controller is further configured to: selectively generate second image data demonstrating a second subset of the source image data including a second portion of the field of view simulating a third inclination angle.
Morales is directed to improved exposure control processes, devices, and systems are provided for wide angle lens imaging systems that suffer from image distortion (abstract) and teaches wherein the controller is further configured to: selectively generate second image data demonstrating a second subset of the source image data including a second portion of the field of view simulating a third inclination angle (Different image data set are generated for different views such as 0 degree – 115 degree. FIG. 1A; The optical center 160 is angularly offset from the longitudinal axis 6 of the endoscope 100 and covers a viewing range 120 of 160 degrees from −45 to +115 degrees relative to the longitudinal axis. Para [0028]- [0029]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shameli to include a plurality of view for collecting different set of data in accordance with the teaching of Morales so that wide range of field could be covered for a wide range of inclination angles.
Regarding claim 10, Shameli, as modified, teaches wherein the first inclination angle is approximately 45 degrees offset from a scope axis of the scope (Morales: The optical center 160 is angularly offset from the longitudinal axis 6 of the endoscope 100 and covers a viewing range 120 of 160 degrees from −45 to +115 degrees relative to the longitudinal axis. Para [0028]- [0029]).
Claim(s) 11, 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shameli (US 20200201022) in view of Morales (US 20170230558).
Regarding claim 11, Shameli as modified teaches wherein the controller is further configured to: selectively output the source data, the first image data, and the second image data to the display screen (Morales: Field of view 102, 106, 104 correspond to different data set; FIG. 4B; The examiner notes that the first image data, and the second image data are not defined in the claim regarding how they are obtained.).
Regarding claim 12, Shameli, as modified, teaches wherein the second inclination angle is 30" and the third inclination angle is 70" (Morales: The optical center 160 is angularly offset from the longitudinal axis 6 of the endoscope 100 and covers a viewing range 120 of 160 degrees from −45 to +115 degrees relative to the longitudinal axis. Para [0028]- [0029]).
Claim(s) 15 is/are rejected under 35 U.S.C. 102 as being anticipated by Shameli (US 20200201022) in view of He (US 20230262347) and further in view of non-patent literature entitled “Generalized dynamic range compression algorithm for visualization of chest CT images.”
Regarding claim 15, Shameli does not expressly disclose wherein the image correction algorithm comprises at least one of an auto-exposure algorithms and a high dynamic range algorithm.
He is directed to apparatus determining a color-skew metric for an image frame captured by an image capture system (abstract) and wherein the image correction algorithm comprises at least one of an auto-exposure algorithms (An auto-exposure algorithm may attempt to brighten up images that look dark by virtue of their skewed chrominance toward a naturally dark-looking color and not by virtue of inadequate illumination of the scene. Para [0023]).
Non-patent literature entitled “Generalized dynamic range compression algorithm for visualization of chest CT images,” is directed to applying algorithm to process images and teaches wherein the image correction algorithm comprises at least one of a high dynamic range algorithm (dynamic range compression (DRC) processing algorithm that can be applied to chest CT images; abstract).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Shameli to include an auto-exposure algorithm so that images may be brighten up after image processing (para [0023]). Further, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify Shameli to include a dynamic range compression algorithm in the image processing algorithm so that high frequency signals can be enhanced (abstract of the non-patent literature).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shameli (US 20200201022) in view of Christian (US 20220079415).
Regarding claim 7, Shameli does not expressly disclose wherein the controller is further configured to: generate a simulated mask enclosed about a perimeter of the first image, wherein the simulated mask includes an orientation cue identifying a direction of the rotation relative to the first image data demonstrating the first portion of the field of view simulating a second inclination angle.
Christian is directed to a visualization system (abstract) and teaches wherein the controller is further configured to: generate a simulated mask (round second image 320; FIG. 6; Para [0176]) enclosed about a perimeter of the first image (image 320 is about the perimeter of the first image 310; FIG.6), wherein the simulated mask includes an orientation cue (viewing axis is pointed by marking 321 in the image 320; Para [0176]) identifying a direction of the rotation relative (The examiner notes that the relative rotation is undefined in the claim.) to the first image data demonstrating the first portion of the field of view simulating a second inclination angle (viewing axis is pointed by marking 321 in the image 320; Para [0176]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shameli to include an image with a cue (such as image 320) in accordance with the teaching of Christian so that a user would know which direction the field of view would be pointing during use of the endoscope.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO – 892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHANKAR R GHIMIRE whose telephone number is (571)272-0515. The examiner can normally be reached 8 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached on 571-272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHANKAR RAJ GHIMIRE/Examiner, Art Unit 3795
/ANH TUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795 02/22/26