DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5 and 8 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 5 and 8 recite “a predetermined threshold”. However both claims 1 and 4 recite “a predetermined threshold”. That’s why the scope of the claim is indefinite.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 5 and 8 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 5 depends on claim 1. However claim 5 doesn’t limit claim 1 as claim 1 also recites “identify areas in which differences in depth along the reference lines are above a predetermined threshold, said areas being confirmed as locations of edges of the at least a portion of the area of interest”.
Claim 8 depends on claim 4. However claim 8 doesn’t limit claim 4 as claim 4 also recites “identify areas in which differences in depth along the reference lines are above a predetermined threshold, said areas being confirmed as locations of edges of the at least a portion of the area of interest”.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 4-5 and 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US Pat. Pub. No. 20160206205, “Wu”) in view of Banerjee et al. (US Patent No. 8238662 “Banerjee”) and Ahmad et al. (US Patent No. 9824260 “Ahmad”).
Regarding claim 1 Wu teaches A system (Fig. 13) for using computer vision to identify locations of edges of an area of interest within a body cavity ([0078] “…. for example. In step S103, the grab cut algorithm is initialized with a rectangle defining the region of interest (ROI). The ROI is defined at an offset of 20 pixels”), comprising:
a camera positionable to capture image data corresponding to a treatment site that includes the area of interest ([0071] “FIG. 2 illustrates a diagram illustrating the flow process of the system according to an embodiment. In the process, an image of the patient is captured by the mobile device 1, thereby generating image information. This information may include information from the imaging sensor 2”);
at least one processor and at least one memory, the at least one memory storing instructions executable by said at least one processor to (Fig. 13 element 1004 and 1003 [0132] “Alternatively, the computer processor can execute a computer program including a set of computer-readable instructions that perform the functions described herein, the program being stored in any of the above-described non-transitory electronic memories and/or a hard disk drive, CD, DVD, FLASH drive or any other known storage media”):
analyze the image data to identify extents of at least a portion of the area of interest within images captured using the camera based on image segmentation techniques ([0029] “…..determine the boundary of the injury portion within the imaging information of the area of interest by utilizing an automatic image segmentation algorithm”);
and determine pixels in the image data at which said extents have been identified (“[0078]….. . The ROI is defined at an offset of 20 pixels from the border. Alternatively, the ROI could be defined as any number of different shapes or wound locations or any offset number of pixels from the border”);
However Wu is silent about receiving user input identifying reference lines cross the extents identified based on image segmentation techniques;
Banerjee teaches receiving user input identifying reference lines cross extents (Col 5 line 64-Col 6 line 3 “Accordingly, horizontal reference lines are identified in the captured image at rows having an intensity profile=1 (step 214), and vertical reference lines are identified in the captured image at columns having an intensity profile=1 (step 216). Each horizontal reference line spans the entire width of the captured image, and each vertical reference line spans the entire height of the captured image”);
Dehmeshki and Wu are analogous art as both of them are related to image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu by receiving user input identifying reference lines cross extents as taught by Banerjee and apply the teaching of Banerjee on the extents identified based on image segmentation techniques of Wu.
The motivation for the above is to provide controllability to the user for identification of width of defect.
Even though Wu as modified by Banerjee teaches obtain depth data from images captured by a camera (“[0095] In step S200 of FIG. 7, depth maps obtained by the structure sensor 3. [0098] In step S203, the depth information within the area, which is represented by the foreground mask is used for further processing. [0100] In step S205, the contour within the foreground mask area is determined and in step S206, the determined contour is projected into 3D space whereby the perimeter is calculated”) but Wu didn’t teach in the same embodiment that this same camera also captures image data.
However Wu teaches in another embodiment that image data and depth data are captured by same camera which is structure 3 ([0063] “…… In an alternative embodiment, imaging information may be obtained from only the structure sensor 3”);
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu as modified by Banerjee by having same camera for capturing image data and depth data based on teaching from alternate embodiment from Wu.
The motivation for the above is to increase efficiency by using single hardware.
Even though Wu as modified by Banerjee teaches analyze the depth data for pixels to identify areas, said areas being confirmed as location of edges of the at least a portion of the area of interest (Wu “Claim 1…….. correlating, using the processing circuitry, the two-dimensional image and the depth map; applying, using the processing circuitry, the boundary of the injury portion designated within the two-dimensional image to the depth map to designate a mask area; determining, using the processing circuitry, characteristics of the injury portion within the mask area based on both the depth map and the two-dimensional image”) but is silent about analyze the depth data for pixels located along the user-defined reference lines crossing the extents identified based on image segmentation techniques to identify areas in which differences in depth along the reference line are above a predetermined threshold;
Ahmad teaches analyze depth data for pixels located along reference lines crossing edges to identify areas in which differences in depth along the reference line are above a predetermined threshold, said areas being confirmed as locations of edges of at least a portion of area of interest (Col 20 lines 5-29 “FIG. 9 will now be used to explain additional details of how to identify, at step 704, one or more spans of pixels that are potentially part of a hole in the subset of pixels corresponding to the user. In general, it is desirable to detect the boundaries of each potential hole. In accordance with specific embodiments, this is accomplished by identifying each horizontal span of pixels (within the subset of pixels specified as corresponding to the user) where on both sides of the horizontal span there is a change in depth values from one pixel to its horizontal neighboring pixel that exceeds a depth discontinuity threshold, as indicated at step 902. Additionally, this is accomplished by identifying each vertical span of pixels (within the subset of pixels specified as corresponding to the user) where on both sides of the vertical span there is a change in depth values from one pixel to its vertical neighboring pixel that exceeds the depth discontinuity threshold. More generally, there is a search for sufficiently large depth discontinuities in each of the two directions. Since an occluding body part is necessarily closer the capture device (e.g., 120) than the occluded body part, depth discontinuities with a positive delta (that exceed the threshold) are identified as a starting point of a potential hole, and depth discontinuities with a negative delta (that exceed the threshold) are identified as an ending point of a potential hole”);
Ahmad and Wu as modified by Banerjee are analogous art as both of them are related to image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu as modified by Banerjee by analyzing the depth data for pixels located along the user-defined reference lines crossing the extents identified based on image segmentation techniques to identify areas in which differences in depth along the reference line are above a predetermined threshold; said areas being confirmed as locations of edges of at least a portion of area of interest similar to analyzing depth data for pixels located along reference lines crossing extents to identify areas in which differences in depth along the reference line are above a predetermined threshold, said areas being confirmed as locations of edges of at least a portion of area of interest as taught by Ahmad.
The motivation for the above is to use a known and standard method of edge detection.
Wu as modified by Banerjee and Ahmad teaches generate an overlay marking the areas confirmed as locations of edges (Wu [0114] “The interface displays the color image 60 and provides the user with the ability to mark the wound 51 by selecting toggle 61 and to mark the background 50 by selecting toggle 62. Button 63 enables the user to erase markings. Once the markings 50 and 51 are placed, the user may select the segmentation button 64, which initiates the processing shown in FIG. 4. Once this processing is complete, the wound border information is returned to the interface to be displayed as border 72.”);
display the overlay over displayed image data (Wu [0114] “….the wound border information is returned to the interface to be displayed as border 72”).
Claim 4 is directed to a method and its steps are similar in scope and functions of the element of the system claim 1 and are also rejected with the same rationale as specified in the rejection of claim 1.
Regarding claims 3 and 7 Wu as modified by Banerjee and Ahmad teaches wherein the camera is a stereoscopic camera (Wu [0064] “….. the structure sensor 3 could also be implemented by a 3D stereo camera”).
Regarding claims 5 and 8 Wu as modified by Banerjee and Ahmad teaches wherein said areas are pixel groups in which differences in depth along the user-defined reference line are above a predetermined threshold, said pixel groups being confirmed as corresponding to locations of edges of the at least a portion of the area of interest (Ahmad Col 20 lines 5-29 “FIG. 9 will now be used to explain additional details of how to identify, at step 704, one or more spans of pixels that are potentially part of a hole in the subset of pixels corresponding to the user. In general, it is desirable to detect the boundaries of each potential hole. In accordance with specific embodiments, this is accomplished by identifying each horizontal span of pixels (within the subset of pixels specified as corresponding to the user) where on both sides of the horizontal span there is a change in depth values from one pixel to its horizontal neighboring pixel that exceeds a depth discontinuity threshold, as indicated at step 902. Additionally, this is accomplished by identifying each vertical span of pixels (within the subset of pixels specified as corresponding to the user) where on both sides of the vertical span there is a change in depth values from one pixel to its vertical neighboring pixel that exceeds the depth discontinuity threshold. More generally, there is a search for sufficiently large depth discontinuities in each of the two directions. Since an occluding body part is necessarily closer the capture device (e.g., 120) than the occluded body part, depth discontinuities with a positive delta (that exceed the threshold) are identified as a starting point of a potential hole, and depth discontinuities with a negative delta (that exceed the threshold) are identified as an ending point of a potential hole”).
Claims 2 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Wu as modified by Banerjee and Ahmad as applied to claim 1 above, and further in view of Regensburger et al. (US Pat. Pub. No. 20200234449, “Regensburger”).
Regarding claims 2 and 6 Wu as modified by Banerjee and Ahmad is silent about image segmentation technique is at least one of color differentiation, brightness differentiation, active contour model, or region growing.
Regensburger teaches image segmentation technique is at least one of color differentiation, brightness differentiation, active contour model, or region growing ([0078] “…. This automatic segmentation on the basis of the different imaging properties of the tissue regions 14 may be based, for example, upon different colors or different contrast properties of the tissue regions 14”);
Regensburger and Wu as modified by Banerjee and Ahmad are analogous art as both of them are related to image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wu as modified by Banerjee and Ahmad by having image segmentation technique is at least one of color differentiation, brightness differentiation, active contour model, or region growing as taught by Regensburger.
The motivation for the above is to use a standard and well known method of image segmentation.
Response to Arguments
Applicant’s arguments, see remarks, filed 09/04/2025, with respect to the rejection(s) of claim(s) 1-4 under 35 USC 112(b) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn.
Applicant’s arguments, see remarks, filed 09/04/2025, with respect to the rejection(s) of under 35 USC 103 have been fully considered and is not persuasive. Therefore, the rejection has been maintained.
Applicant argues see remarks page 2 “As previously argued, Applicant respectfully submits that proper motivation does not exist for the cited combination of Wu and Banerjee. The Office Action asserts that one skilled in the art would be motivated to import the gridlines of Banerjee into Wu “to provide controllability to the user for identification of width of defect.” However, this motivation is not reasonable given the teachings of the references”.
Examiner replies, the motivation for combining references doesn’t have to come from the reference. In response to applicant’s argument that there is no motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, teaching is of “receiving user input identifying reference lines across edges” is from the reference Banerjee and the motivation is from the knowledge generally available to one of ordinary skill in the art, because an ordinary skilled person in the art knows the a reference line or grid line provides a way to measure width of an object.
Applicant argues see remarks page 2 “Banerjee describes gridlines generated based on luminance profiles for the purpose of region manipulation, not measurement. There is no discussion in Banerjee that the disclosed gridlines are used to identify regions for measurement or to confirm anatomical edges. In contrast, Wu describes two distinct methods for measuring wound dimensions: (a) using Principal Component Analysis (PCA) applied to a point cloud, and (b) using a rotated rectangle enclosing the wound (see Wu {[0093] and Fig. 12)…. The Examiner’s rationale for combining these references lacks a reasonable expectation of success and would result in a system that is less accurate and less suited to the surgical context”.
Examiner replies, Wu uses two different methods of measuring width and length. But it doesn’t mean that an easier method of measuring of width and length ( by using gridlines or reference lines) of an wound can’t be proposed. Examiner didn’t propose to apply the teaching of Banerjee after applying Wu’s own method of measuring depth and length. Examiner proposed to add another method of measuring width and length based on Banerjee. In the argument, applicant tries to bodily incorporate the teaching of Banerjee to Wu reference. In response to applicant's argument that, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
Applicant argues see remarks pages 2-3 “Applicant acknowledges that the test for obviousness is not limited to bodily incorporation of one reference into another, and that combinations may be proposed to yield alternative approaches. However, the Examiner’s rationale still lacks a reasonable expectation of success. Wu’s methods for measuring wound dimensions—Principal Component Analysis and rotated rectangles—are specifically designed to account for wound geometry that is not aligned with vertical or horizontal axes. Banerjee’s gridlines, by contrast, are aligned to rows and columns and are not adaptable to irregular wound shapes. Adding Banerjee’s gridlines as an alternative measurement method would not only be incompatible with Wu’s approach, but would also lead to inaccurate measurements, as the maximum extents of the wound would not be captured. This is not a matter of bodily incorporation, but of technical incompatibility and lack of motivation to combine. The proposed combination would degrade the accuracy of Wu’s system, which is contrary to the goals of both references”.
Examiner replies, Banerjee is integrated to provide user selection of reference lines cross extents. This would provide advantage of working with selected reference lines instead of performing search on whole image data. Banerjee’s inclusion will also provide control to the user to select reference lines on his choice. This is the motivation for including Banerjee’s selection of gridlines. In response to applicant’s argument that maximum extent of the wound would not be captured is not correct because Banerjee selects reference lines that covers entire image in horizontal and vertical directions. Banerjee, Col 5 line 64-Col 6 line 3 “…..Each horizontal reference line spans the entire width of the captured image, and each vertical reference line spans the entire height of the captured image”. Therefore applicant’s argument u not persuasive.
Applicant argues see remarks page 3 “Furthermore, the Examiner’s interpretation of Wu’s Claim 1 as teaching “confirmation of edges” is not supported by the Wu specification. Wu describes mapping a 2D boundary to a depth map and determining characteristics such as width and depth. These characteristics are quantitative measurements, not qualitative confirmations of edge presence. The claimed invention, by contrast, performs a depth-based validation of segmentation-identified edges, using user-defined reference lines and a threshold-based analysis of depth discontinuities. This process is not taught or suggested by Wu, Banerjee, or Ahmad”.
Examiner replies, Banerjee teaches user-defined reference lines cross extents (Col 5 line 64-Col 6 line 3 “Accordingly, horizontal reference lines are identified in the captured image at rows having an intensity profile=1 (step 214), and vertical reference lines are identified in the captured image at columns having an intensity profile=1 (step 216). Each horizontal reference line spans the entire width of the captured image, and each vertical reference line spans the entire height of the captured image”) and Wu’s claim 1 teaches, analyze the depth data for pixels to identify areas, said areas being confirmed as locations of edges of the at least a portion of the area of interest (Wu “Claim 1…….. correlating, using the processing circuitry, the two-dimensional image and the depth map; applying, using the processing circuitry, the boundary of the injury portion designated within the two-dimensional image to the depth map to designate a mask area; determining, using the processing circuitry, characteristics of the injury portion within the mask area based on both the depth map and the two-dimensional image”. Wu’s claim 1 uses depth data and 2D image data to designate/identify a mask area which is the claimed “said area” and that said area is confirmed to corresponding to the edge or boundary of the injury area, but is silent about analyze the depth data for pixels located along the reference lines crossing the extents identified based on image segmentation techniques to identify areas in which differences in depth along the reference line are above a predetermined threshold;
Ahmad (Col 20 lines 5-29) teaches analyze depth data for pixels located along reference lines crossing extents to identify areas in which differences in depth along the reference line are above a predetermined threshold, said areas being confirmed as locations of edges of at least a portion of area of interest.
Examiner proposed to modify Wu as modified by Banerjee by analyzing the depth data for pixels located along the user-defined reference lines crossing the extents identified based on image segmentation techniques to identify areas in which differences in depth along the reference line are above a predetermined threshold similar to analyzing depth data for pixels located along reference lines crossing extents to identify areas in which differences in depth along the reference line are above a predetermined threshold, said areas being confirmed as corresponding to edges of at least a portion of area of interest as taught by Ahmad. The motivation for the above is to use a known and standard method of edge detection.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAPTARSHI MAZUMDER whose telephone number is (571)270-3454. The examiner can normally be reached 8 am-4 pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at (571)272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAPTARSHI MAZUMDER/Primary Examiner, Art Unit 2612