DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1, 3-6, and 8-10 are pending in this application. Claims 2 and 7 are cancelled, and Claims 1, 3-6, and 8-10 have been examined on the merits.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3 and 6, 8 are rejected under 35 U.S.C. 103 as being unpatentable over Teixeira (US20210110594A1) in view of Shachaf (US20140350395A1).
Regarding 1,
Teixeira teaches an animal in-vivo imaging device comprising: a camera for capturing an image of an animal (corresponding disclosure in at least [0032], where a camera is used for capturing medical images “a depth camera performs act 10. A computer, such as for a medical imaging system, performs acts 12 and/or 16. A display performs act 14” and further in [0024], where it’s specified the human body (animal) is being imaged “The internal anatomy of a human body is estimated from the surface data”)
a three-dimensional scanner for measuring three-dimensional shape information of the animal (corresponding disclosure in at least [0024], where there is a scanner (3D CT representation) getting 3D shape information of the patient “The internal anatomy of a human body is estimated from the surface data. A 3D CT representation is predicted from patient surface data”; the scanner is one of the cameras from [0033]);
and a processor configured to: output depth information and horizontal position information of a target organ by processing an input comprising a type of animal, the target organ, a preview image captured by the camera, and three-dimensional shape information measured by the three-dimensional scanner into a trained neural network model, wherein the depth information is a vertical distance between a surface of the animal and the target organ (corresponding disclosure in at least [0014], where there is a CT volume generated (depth information) based on inputs of the patient, which would be included within the “The landmarks, segmentation, or other parameterization may be used, or organ location is detected from the 3D CT representation”; the type of animal is implicitly human, full-body anatomy is utilized which includes a target organ [0034], and additional camera images are utilized as in [0033] which can be considered preview images and further in [0058], where a neural network is used “ the image processor generates the 3D CT representation of the patient by a machine-learned generative network (e.g., generator 56) in response to input of the surface data 50 and the segmentation and/or landmark location 55 to the machine-learned generative network. The parameters 55 are input to the Conditional Generation Network (CGN), which generates the 3D CT representation. The new 3D CT representation 57 can then go back to the parameters neural network (PNN or generator 54) in the iterative pipeline” and further in [0051], where there is a generator that segments the anatomy or organ, with landmarks or reference points “The generator 54 is a parameterization network trained to parameterize, such as generate a segmentation (e.g., voxel labels or mesh of a surface for an organ, tissue, anatomy, or internal system in 3D), landmarks (e.g., one or more anatomical reference points positioned in 3D), or another parameterization”, and that information gathered is inputted into another generator (training data) with the information “The generator 54 receives a 3D CT representation 53 or 57 and the surface data 50. In response to input, the generator 54 outputs a representation of the location or locations of internal anatomy, such as segmentation, landmarks, or another parameterization” which implicitly include position information in all cartesian directions, e.g. horizontal, vertical, x, y, z coordinates).
wherein the neural network model is trained by using a training dataset which uses, as inputs,
the type of animal, the target organ, the captured image of the animal, and the three-dimensional shape information, and generates an output comprising the depth information and the horizontal position information of the target organ (corresponding disclosure in at least [0014], where there is a CT volume generated (depth information) based on inputs of the patient, which would be included within the “The landmarks, segmentation, or other parameterization may be used, or organ location is detected from the 3D CT representation”; the type of animal is implicitly human, full-body anatomy is utilized which includes a target organ [0034], and additional camera images are utilized as in [0033] which can be considered preview images).
Teixeira does not teach a focus lens for focus adjustment and a focus adjustment unit configured to adjust a focus by controlling a focus driving motor which drives the focus lens according to the depth information.
Shachaf, in a similar field of endeavor, teaches a similar concept (structure imaging), of a focus lens for focus adjustment (corresponding disclosure in at least [0100], where there is a lens for focusing on the camera “A consumer or professional digital single-lens reflex (DSLR) camera is one example of an integrated imaging system…. The lens may have macro-focusing capability”)
and a focus adjustment unit configured to adjust a focus by controlling a focus driving motor which drives the focus lens according to the depth information (corresponding disclosure in at least [0225], where with the focus lens, there is an autofocus method that will focus depending on the lens’ distance from the subject (area of interest), which is associated with a particular depth/distance “Auto-focus becomes more critical or useful when: (a) the numerical aperture is larger, (b) the lens is closer to the subject, (c) the magnification is higher, or (d) the resolution is higher”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the focus lens and controlling the focus as taught by Shachaf. One of the ordinary skill in the art would have been motivated to incorporate this because focusing ensures a clear image even as the camera or imaging device is moved, allowing for precise boundaries when completing analysis.
Regarding Claim 3, Teixeira and Shachaf teach the limitations of Claim 1, and Teixeira further teaches wherein the horizontal position information is displayed in the captured image of the animal (corresponding disclosure in at least [0051], where there is a generator that segments the anatomy or organ, with landmarks or reference points and captures the image (output of image) “The generator 54 is a parameterization network trained to parameterize, such as generate a segmentation (e.g., voxel labels or mesh of a surface for an organ, tissue, anatomy, or internal system in 3D), landmarks (e.g., one or more anatomical reference points positioned in 3D), or another parameterization”, and that information gathered is inputted into another generator (training data) with the information “The generator 54 receives a 3D CT representation 53 or 57 and the surface data 50. In response to input, the generator 54 outputs a representation of the location or locations of internal anatomy, such as segmentation, landmarks, or another parameterization” which implicitly include position information in all cartesian directions, e.g. horizontal, vertical, x, y, z coordinates).
Regarding Claim 6,
Teixeira teaches an operating method of an animal in-vivo imaging device including a camera for capturing an image of an animal (corresponding disclosure in at least [0032], where a camera is used for capturing medical images “a depth camera performs act 10. A computer, such as for a medical imaging system, performs acts 12 and/or 16. A display performs act 14” and further in [0024], where it’s specified the human body (animal) is being imaged “The internal anatomy of a human body is estimated from the surface data”)
and a three-dimensional scanner for measuring three-dimensional shape information of the animal (corresponding disclosure in at least [0024], where there is a scanner (3D CT representation) getting 3D shape information of the patient “The internal anatomy of a human body is estimated from the surface data. A 3D CT representation is predicted from patient surface data”), comprising:
processing an input comprising a type of animal, a target organ of the animal, a preview image of the animal captured by the camera,
and three-dimensional shape information of the animal measured by the three-dimensional scanner using a neural network model to output depth information and horizontal position information of the target organ (corresponding disclosure in at least [0014], where there is a CT volume generated (depth information) based on inputs of the patient, which would be included within the “The landmarks, segmentation, or other parameterization may be used, or organ location is detected from the 3D CT representation”; the type of animal is implicitly human, full-body anatomy is utilized which includes a target organ [0034], and additional camera images are utilized as in [0033] which can be considered preview images and further in [0051], where there is a generator that segments the anatomy or organ, with landmarks or reference points “The generator 54 is a parameterization network trained to parameterize, such as generate a segmentation (e.g., voxel labels or mesh of a surface for an organ, tissue, anatomy, or internal system in 3D), landmarks (e.g., one or more anatomical reference points positioned in 3D), or another parameterization”, and that information gathered is inputted into another generator (training data) with the information “The generator 54 receives a 3D CT representation 53 or 57 and the surface data 50. In response to input, the generator 54 outputs a representation of the location or locations of internal anatomy, such as segmentation, landmarks, or another parameterization” which implicitly include position information in all cartesian directions, e.g. horizontal, vertical, x, y, z coordinates);
wherein the neural network model is trained by using a training dataset which uses, as inputs, the type of animal, the target organ, the captured image of the animal, and the three-dimensional shape information (corresponding disclosure in at least [0014], where there is a CT volume generated (depth information) based on inputs of the patient, which would be included within the “The landmarks, segmentation, or other parameterization may be used, or organ location is detected from the 3D CT representation”; the type of animal is implicitly human, full-body anatomy is utilized which includes a target organ [0034], and additional camera images are utilized as in [0033] which can be considered preview images),
and generates an output, comprising the depth information and horizontal position information of the target organ (corresponding disclosure in at least [0077] ,where the output is a position or depth of the organ “The image of the 3D CT representation shows the patient shape as well as positions of one or more organs” and further in [0051], where there is a generator that segments the anatomy or organ, with landmarks or reference points “The generator 54 is a parameterization network trained to parameterize, such as generate a segmentation (e.g., voxel labels or mesh of a surface for an organ, tissue, anatomy, or internal system in 3D), landmarks (e.g., one or more anatomical reference points positioned in 3D), or another parameterization”, and that information gathered is inputted into another generator (training data) with the information “The generator 54 receives a 3D CT representation 53 or 57 and the surface data 50. In response to input, the generator 54 outputs a representation of the location or locations of internal anatomy, such as segmentation, landmarks, or another parameterization” which implicitly include position information in all cartesian directions, e.g. horizontal, vertical, x, y, z coordinates).
Teixeira does not teach a focus lens for focus adjustment and adjusting a focus by controlling a focus driving motor which drives the focus lens according to the depth information.
Shachaf, in a similar field of endeavor, teaches a similar concept (structure imaging), of a focus lens for focus adjustment of the camera(corresponding disclosure in at least [0100], where there is a lens for focusing on the camera “A consumer or professional digital single-lens reflex (DSLR) camera is one example of an integrated imaging system…. The lens may have macro-focusing capability”)
and a focus adjustment unit configured to adjust a focus by controlling a focus driving motor which drives the focus lens according to the depth information (corresponding disclosure in at least [0225], where with the focus lens, there is an autofocus method that will focus depending on the lens’ distance from the subject (area of interest), which is associated with a particular depth/distance “Auto-focus becomes more critical or useful when: (a) the numerical aperture is larger, (b) the lens is closer to the subject, (c) the magnification is higher, or (d) the resolution is higher”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the focus lens and controlling the focus as taught by Shachaf. One of the ordinary skill in the art would have been motivated to incorporate this because focusing ensures a clear image even as the camera or imaging device is moved, allowing for precise boundaries when completing analysis.
Regarding Claim 8, the combined references of Teixeira and Shachaf teach the limitations of Claim 7, and Shachaf further displaying and outputting the position information in the captured image in which the focus is adjusted by the adjusting of the focus (corresponding disclosure in at least [0106], where there is a focus lens which adjusts the focus of the outputted image “a macro lens, macro focus, or macro imaging refers to the having a visible resulting image... if imaging a patient mole whose actual diameter is one millimeter, a macro image could be any image of that mole displayed with a visible diameter of at least a one-millimeter.”).
Claims 4 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Teixeira (US20210110594A1) and Shachaf (US20140350395A1) as taught in Claims 1 and 6, and in further view of Ren (US20230065399A1).
Regarding Claim 4 and 9, Teixeira and Shachaf teach the limitations of Claims 1 and 6 and further teach wherein, when a part which appears fluorescent in the captured image and the horizontal position information (corresponding disclosure in at least [0051], where there is a generator that segments the anatomy or organ, with landmarks or reference points and provides horizonal position information “The generator 54 is a parameterization network trained to parameterize, such as generate a segmentation (e.g., voxel labels or mesh of a surface for an organ, tissue, anatomy, or internal system in 3D), landmarks (e.g., one or more anatomical reference points positioned in 3D), or another parameterization”, and that information gathered is inputted into another generator (training data) with the information “The generator 54 receives a 3D CT representation 53 or 57 and the surface data 50. In response to input, the generator 54 outputs a representation of the location or locations of internal anatomy, such as segmentation, landmarks, or another parameterization” which implicitly include position information in all cartesian directions, e.g. horizontal, vertical, x, y, z coordinates) do not match, the neural network model uses additional training data which has additional horizontal position information of the part which appears fluorescent (corresponding disclosure in at least [0287] of Shachaf, where information is taken to match (align) “Finally, the images are presented to the medical professional 49. Ideally the visible light image and the fluorescent light image are presented as an overly, where the medical professional, using a slider or similar means, can dynamically change the overlay from 100% one image to 100% the other image as a way to easily see how the two images align. Also, matching images from the library are presented, along with quantitative matching coefficients and information about the library images” and further in [0286] of Shachaf, where a machine learning model is used for this method “ classification 48 is predominantly within the mole border. Supervised machine learning is performed on the library of images.”), but does not teach retraining.
Ren, in a similar field of endeavor, teaches a similar concept (scanning) of retraining a neural network model with additional training data ( corresponding disclosure in at least [0145], where there is a retraining portion “a machine learning model may then be retrained, or updated, at any number of other facilities, and a retrained or updated model may be made available in model registry”)
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated the retraining unit for retraining as taught by Ren. One of the ordinary skill in the art would have been motivated to incorporate this because when the model encounters significant errors or discrepancies when compared to the true data, it can be updated to be more accurate.
Claims 5 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Teixeira (US20210110594A1) and Shachaf (US20140350395A1) as taught in Claims 1 and 6, and in further view of Shibasaki (JP2021144359A).
Regarding Claim 5 and 10, the combined references of Teixeira and Shachaf teach the limitations of Claim 1 and 6, and inputting into the neural network model (corresponding disclosure in at least [0063], where there is a neural network model and information being inputted for training data “many samples of training data (e.g., surface data, ground truth parameterization (e.g., segmentation or landmarks), and ground truth 3D CT representation) are used to learn to output the 3D CT representation from input surface data”), but does not specify the posture of the animal as an input.
Shibasaki, in a similar field of endeavor, teaches a similar concept (machine learning with images), of the posture of the animal (corresponding disclosure in at least [0046], where the posture of the person (animal) is taken as an input “position estimation unit 23 can estimate the position of the left shoulder of the person in the captured image because the depth distribution of the region estimated by the person detection unit 22 is similar to the depth distribution of the identification model for position estimation generated from the three dimensional model of the same posture”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated receiving the posture of an animal as an input as taught by Shibasaki. One of the ordinary skill in the art would have been motivated to incorporate this because it’s another parameter used in the machine learning model for determining the location of an organ. Taking into account posture would increase accuracy of the determination since it patient may move from the default position.
Response to Arguments
Applicant’s arguments regarding the 35 U.S.C. 112b rejection filed 12/18/2025 have been fully considered and are withdrawn in light of the amendments.
Applicant's arguments filed 12/18/2025 regarding the 35 U.S.C 103 rejections have been fully considered but they are not persuasive.
Regarding Claim 1 (and similarly Claim 6), Applicant argues that Teixeira does not teach outputting or generating the depth information and the horizontal position information using the neural network model. However, Teixeira teaches the neural network model ([0058], see Office Action above) and further teaches the generation of depth information by using depth cameras and sensors ([0037] of Teixeira further highlights the use of a model displaying depth “ a statistical shape model is fit to the depths”). Further, [0051] implicitly includes position information in the cartesian directions , e.g. horizontal, vertical, x, y, z coordinates through the parametrization network. Further, as stated in the Office Action above, the type of animal is implicitly human, full-body anatomy is utilized which includes a target organ [0034], and additional camera images are utilized as in [0033] which can be considered preview images, all which are included as an input for the model. All other claims are rejected due to their dependency to the independent claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN KIM whose telephone number is (571)272-1821. The examiner can normally be reached Monday-Friday 6-2 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached at (571) 270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.E.K./ Examiner, Art Unit 3797
/SERKAN AKAR/Primary Examiner, Art Unit 3797