Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/17/26 has been entered.
Response to Arguments
Applicant's arguments directed to the previously applied combination have been fully considered. The arguments are moot in view of the modified grounds of rejection below.
Applicant emphasizes that claims 1 and 16 require applying the deep neural network to the functional movement of the human subject “to select” the updated segmentation algorithm
Aguiar teaches a learned selector that automatically selects the most suitable image segmentation algorithm among multiple candidates (Abstract).
For the emphasis-guidelines, validating, and updating limitations of claim 9, please see newly cited Bhanu (5048095),
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 16-20 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor regards as the invention.
Regarding claim 16, the limitation “the updated segmentation model” lacks antecedent basis. Claim 16 recites “an updated segmentation algorithm” but does not previously recite an updated segmentation model.
Claims 17-20 depend from claim 16.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kanaujia (“Part Segmentation of Visual Hull for 3D Human Pose Estimation”) in view of Hasler (20140072175 A1) in view of Aguiar (“A meta-learning approach for selecting image segmentation algorithm”) in further view of Yi (US 20190073524 A1).
Regarding claim 1, Kanaujia teaches an image segmentation method for analyzing digital image data and detecting a location of a joint center of a human subject, the method comprising the steps of capturing a digital image using an image capture device (abstract; Section 4, image/voxel data of a human subject),
detecting a visual hull segmentation of the digital image (Section 4, p. 544, segmenting the voxels in the visual hull to different body segments), the visual hull segmentation including one or more proxy spheres (Hasler teaches representing the human body with a set of 3D spatial Gaussians, i.e., proxy spheres attached to the skeleton, pars. 9, 11 and 31);
selecting an initial segmentation algorithm based on a context of the visual hull segmentation (Kanaujia teaches a per-part classifier as the initial segmentation algorithm, Section 4.1; Trautwein teaches selecting the algorithm based on the context of the image content, pars. 56-59);
identifying a set of initial joint center coordinates using the initial segmentation algorithm (Kanaujia, Section 6 and Fig. 1(e), mean shift clustering on the part-classifier probabilities localizes the joint centers);
capturing a functional movement of the human subject (Kanaujia, Section 7, jogging and walking sequences of subjects S1, S2 and S3);
identifying a set of updated joint center coordinates using the updated segmentation algorithm (Kanaujia, Section 6, mean shift localization using the selected part-segmentation classifier);
and generating an updated segmentation model based on the set of updated joint center coordinates (Kanaujia, Section 6, coarse body model fit to the localized joint centers).
Kanaujia does not expressly teach the visual hull segmentation including one or more proxy spheres. Hasler teaches one or more proxy spheres representing the human body (Hasler, pars. 9, 11 and 31, a human model ).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to modify Kanaujia to include the proxy-sphere body representation of Hasler. The reason is to provide spatial body-part proxies associated with the skeletal joints that can be fit to image data and used to represent and track the human body during joint localization.
Kanaujia and Hasler does not expressly teach selecting an initial segmentation algorithm based on a context of the visual hull segmentation or selecting an updated segmentation algorithm.
Aguiar teaches selecting a segmentation algorithm from among multiple candidate segmentation algorithms (Aguiar, Abstract, meta-learning used to select/recommend the most suitable image segmentation algorithm from among eight segmentation algorithms for a new dataset).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to modify the combination of Kanaujia and Hasler to use Aguiar’s learned segmentation-algorithm selector to select the segmentation algorithm based on Kanaujia’s visual-hull context information and to select, from the available part-segmentation algorithms. The reason is to reduce the computational cost of determining a suitable algorithm and improving segmentation performance.
The combination of Kanaujia, Hasler and Aguiar does not expressly teach applying a deep neural network algorithm to the functional movement of the human subject to select the updated segmentation algorithm.
Yi teaches applying a deep neural network algorithm to the functional movement of a human subject (Yi, pars. 166 and 228, inputting an offset matrix representing walking-behavior information into a DNN).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Kanaujia, Hasler and Aguiar the ability to utilize the movement information produced by Yi’s deep neural network as selection information. The reason is for improving the resulting part segmentation and joint-center localization.
Regarding claim 2, see the abstract of Kanaujia, image streams (video).
Regarding claim 3, see Hasler pars. 3, 10, 68 and 72, a 3D markerless motion capture device.
Regarding claim 4, see Hasler pars. 9, 31 and 62, the proxy spheres (Gaussian blobs) represent body segments of the human subject.
Regarding claim 5, see Kanaujia Section 6 and Section 4.1, initial joint centers determined from the segmented visual hull.
Regarding claim 6, see Yi pars. 166 and 228 (deep neural network operating on the movement) together with Aguiar (Abstract) as combined in the rejection of claim 1; the network utilizes the functional movement and at least one visual hull segmentation algorithm to select the updated segmentation algorithm.
Regarding claim 7, wherein the set of updated joint center coordinates of the human subject identified using the updated segmentation model are an improved representation of the human subject’s joint center locations compared to the set of initial joint center coordinates of the human subject identified using the initial segmentation algorithm (Kanaujia, Section 7 and Table 2, skeleton fitting improves joint-estimation accuracy; Aguiar, Abstract, selecting the segmentation algorithm identified as most suitable for the data).
Regarding claim 8, training a segmentation algorithm based on a target anatomical structure (Kanaujia, Sections 4.1 and 5, dividing the human body into 31 body-part segments, generating labeled training data for the respective parts, and training one-against-rest classifiers for each part); and
segmenting the target anatomical structure of the human subject using the segmentation algorithm trained for the target anatomical structure (Kanaujia, Sections 4.1 and 5 and Fig. 1(c)-(d), using the trained part classifiers to segment visual-hull voxels into the corresponding body-part segments).
Regarding claims 16-20, see the rejection of claim 1 and 4-6.
Claims 9-15 are rejected under 35 U.S.C. 103 as being unpatentable over Kanaujia in view of Hasler in view of Aguiar in view of Yi in further view of Bhanu (5048095).
Regarding claim 9, see the rejection of claim 1 and in addition Bhanu teaches validating the applying of the deep neural network algorithm to the functional movement (col. 8, evaluation component and reward),
updating the baseline emphasis guidelines based on the validating step and saving as updated emphasis guidelines (col. 8-9, based on the evaluated segmentation quality, updating and retaining the global population),
selecting an updated segmentation algorithm using the updated emphasis guidelines (figs. 6 and 8, stored segmentation parameter and col. 8-9, using the updated selection for the next selection),
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Kanaujia, Hasler, Aguiar, and Yi the ability to evaluate the resulting segmentation quality and, based on that evaluation, update and store the selection knowledge used to select the segmentation configuration, as taught by Bhanu. The reason is to allow the selection to adapt from measured segmentation performance, thereby improving joint-center localization accuracy over successive processing.
Regarding claim 10, see Hasler pars. 19 and 33-34, the digital image is a 3D image.
Regarding claim 11, see the rejection of claim 3, a markerless motion capture device.
Regarding claim 12, see Kanaujia Section 6 and Section 4.1, initial joint centers determined based on the visual hull segmentation.
Regarding claim 13, see the rejection of claim 4, the proxy spheres represent body segments (Hasler pars. 9, 31).
Regarding claim 14, see Hasler pars. 27, 28-37, the updated segmentation model can identify distinct body segments (58 joints / 63 Gaussians attached to distinct segments).
Regarding claim 15, validating the deep neural network algorithm further comprises receiving image data from the image capture device (Kanaujia, Section 4), calculating segmentations from the initial segmentation algorithm to establish the baseline emphasis guidelines (Bhanu, col. 4-7, building the fitness-weighted knowledge base from segmentation results), creating training data including a digital library of human subjects and associated functional movement data with skeletal tracking (Kanaujia, Section 7, HumanEva dataset of subjects S1-S3 with jogging/walking sequences and ground-truth skeletal/joint tracking), and confirming the deep neural network algorithm selects a segmentation algorithm that accurately identifies a joint center localization (Bhanu, col. 8, accepting a selected configuration only when its evaluated segmentation quality exceeds a predefined threshold, in combination with Kanaujia's evaluation of joint localization accuracy, Section 7).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: O'Reilly (20160132787) teaches a multi-armed-bandit selector whose per-arm selection scores are updated from evaluated model performance and stored, so the selection policy improves over time. Muncaster (20220301274) teaches a system that selects and allocates the image input among a plurality of neural network models to optimize processing.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HADI AKHAVANNIK whose telephone number is (571)272-8622. The examiner can normally be reached 9 AM - 5 PM Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HADI AKHAVANNIK/Primary Examiner, Art Unit 2676
Prosecution Insights