CTNF 18/838,845 CTNF 83206 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Status of claims: claims 1-14 are pending below. Information Disclosure Statement 06-52 The information disclosure statement (IDS) submitted on August 115, 2024 was filed and considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. 07-30-06 This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “an acquisition unit” in claim 1, “a behavior output unit” in claim 1, “a basis output unit” in claim 1, “a presentation unit” in claim 1, “a storage unit” in claim 1, “the basis unit” in claim 5, “the presentation unit” in claim 6, “the storage unit” in claim 8, “the storage unit” in claim 9, “the storage unit” in claim 10, “the storage unit” in claim 11, “the storage unit” in claim 12. Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim element “program” is viewed as non-statutory subject matter. A review of the specification of the instant invention mentioned in paragraph 0134-0138, specifically 0135 recited “In addition, any non-transitory computer-readable storage medium may be used” open the possibility where non-transitory computer-readable storage medium may not be embodied, which would include signal which is view as non-statutory subject matter. Examiner advise amending to “non-transitory computer-readable storage medium” to overcome the rejection. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-3 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over SUZUKI et al (US 2023/0282345) in view of Junio et al (US 2022/0218421) . Claim 1, similarity claim 13 and 14: SUZUKI et al (US 2023/0282345) teaches the following subject matter: An information processing apparatus, comprising: an acquisition unit that acquires a treatment image related to treatment (figures 8-9 and 0121-0124 detail use of endoscope 110 (acquisition unit) image of operative field image with grasper and scissors (treatment) used during treatment (surgical operation)) ; and a basis region indicating position information of a basis on which the behavior information is output, by inputting the treatment image to each of a plurality of recognizers (figures 8-9 and 0121-0124 specifically 0122 with image recognizer 501 (recognizers) for grasper and scissors and their position around the gaze point) ; a basis output unit that outputs basis information related to the basis by inputting the treatment image cropped on the basis of the basis region to a classifier (figures 8-9 and 0121-0124, specifically 0123 detail picture-in-picture (PinP, cropped, target portion/area) reflecting the attention to the sub display of treatment area; 0013 detail used of machine learning model (classifier) to output basis information on the target portion; 0063 detail portion processed by machine learning model outputting analysis information, score, certain factor influence, cause, certain result) ; a presentation unit that presents a plurality of pieces of the behavior information and a plurality of pieces of the basis information to the user (figure 9 and 0123-0124 detail display device 149 for the surgeon or doctor (user) performing (treatment) checking information (plurality of pieces as mentioned above) on the control system) ; and a storage unit that stores input information input by the user on a basis of the behavior information and the basis information, as learning data of the plurality of recognizers and the classifier (0193 detail adding data (all factors mentioned above) to database (storage unit) for relearning the learner, and learner is updated (step S2405) where database may be external database) . SUZUKI et al teaches all the subject matta above, but not the following: a behavior output unit that outputs behavior information related to a behavior of a medical device related to the treatment. Junio et al (US 2022/0218421) teaches the following subject matter: a behavior output unit that outputs behavior information related to a behavior of a medical device related to the treatment (0098 detail behavior output where their specific pattern is recognized from different surgical tools traverse different tissues with machine learning). SUZUKI et al and Junio et al are both in the field of image analysis, especially the use of machine learning during surgical treatment to assist user/surgeon in for tracking and labeling of surgical tools and organ during operation such that the combine outcome is predictable. Therefore it would have been obvious to one having ordinary skill before the effective filing date to modify SUZUKI et al by Junio et al regarding behavior analysis further assist in defining the expected parameter measurements as the tool traverses a given anatomical feature further allowing a greater definition of the expected measurements for operation of specific tools as disclosed by Junio et al in paragraph 0098. Regarding method of claim 13, SUZUKI et al teaches flowchart/method in figures 13, 15, 20 and 22. Regarding program of claim 14, SUZUKI et al teaches CPU and local memory for program for operation in 0076. Claim 2: SUZUKI et al teach: The information processing apparatus according to claim 1, wherein the behavior information includes at least one of position information, movement information, or motion information of the medical device (0063 detail information such as motion and position information or medical robot device) . Claim 3: SUZUKI et al teach: The information processing apparatus according to claim 1, wherein the basis information includes the basis and a cause of lowering an accuracy of the basis (figure 3 and 0089-0091 detail part of organ hidden by surgical tools cannot be observed as data, therefore decrease accuracy; 0164-0166 detail the presentation of uncertainty/lack of data, unknow environment/conditions that are smooth out by plurality of forms of neural networks in paragraph 0166) , the basis includes at least one of a surgical tool, a shaft of the surgical tool, or an organ (figure 9 detail grasper and scissors; 0085 detail environment recognition information including depth, map, arrangement of organs and instruments) , and the cause includes at least one of smoke, dirt of the surgical tool, dirt of a lens, or occlusion (figure 3 and 0089-0091 detail lens observation for imaging, where 0134 further detail occlusion or the likes) . 07-22-aia AIA Claim s 4-8 and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over SUZUKI et al (US 2023/0282345) in view of Junio et al (US 2022/0218421) as applied to claim 1 above, and further in view of Reiter et al (US 2015/0297313) . Claim 4: SUZUKI et al and Junio et al teaches all the subject matter above. SUZUKI et al and Junio et al do not teaches the following subject matter: The information processing apparatus according to claim 1, wherein the plurality of recognizers includes a first recognizer that performs offline learning, and a second recognizer that performs online learning, the first recognizer outputs first behavior information and a first basis region, and the second recognizer outputs second behavior information and a second basis region. Reiter et al (US 2015/0297313) teaches the following subject matter: The information processing apparatus according to claim 1, wherein the plurality of recognizers includes a first recognizer that performs offline learning (0034 detail off-line learned to label input scene image 201 for anatomical scene 202 and medical tool(s) 203 and 204) , and a second recognizer that performs online learning, the first recognizer outputs first behavior information and a first basis region, and the second recognizer outputs second behavior information and a second basis region (0069 detail online for class-labeled feature such as tools and position of each time in each frame location) . SUZUKI et al and Reiter et al are in the field of image analysis, especially the use of machine learning during surgical treatment to assist user/surgeon in for tracking and labeling of surgical tools and organ during operation such that the combine outcome is predictable. Therefore it would have been obvious to one having ordinary skill before the effective filing date to modify SUZUKI et al by Reiter et al with the use of off-line and on-line, such the online/remote center will remain constant across frame accumulating and provide/achieve a stable solution as disclosed by Reiter et al in 0069. Claim 5: SUZUKI et al teach: The information processing apparatus according to claim 4, wherein the basis output unit outputs first basis information and second basis information by inputting a first treatment image cropped on a basis of the first basis region and a second treatment image cropped on a basis of the second basis region to the classifier (figures 8-9 and 0121-0124, specifically 0123 detail picture-in-picture (PinP, cropped, target portion/area) into classifier 0013, where figure 17 detail other target portion/region such as Grasper, Liver and Scissors) . Claim 6: SUZUKI et al teach: The information processing apparatus according to claim 5, wherein the presentation unit presents, to the user, a graphical user interface (GUI) in which the first behavior information (figure 9 and 0123-0124 detail display device 149 for the surgeon or doctor (user)) , the second behavior information, the first basis information, and the second basis information can be recognized (figure 17 detail other target portion/region such as Grasper, Liver and Scissors are each processed for their basis information) . Claim 7: SUZUKI et al teach: The information processing apparatus according to claim 6, wherein the input information includes at least one of a selection of the behavior information, a selection of the determination basis, an input of new behavior information different from the behavior information, an input of new basis information different from the basis information, OR a new cropped treatment image different from the cropped treatment image (figure 17 detail other target portion/region such as Grasper, Liver and Scissors are each processed for their basis information) . Claim 8: SUZUKI et al teach The information processing apparatus according to claim 7, wherein if the first behavior information or the second behavior information is correct, the storage unit stores the first behavior information or the second behavior information that is selected by the user via the GUI, as learning data of the plurality of recognizers and the classifier (0002 detail evaluation data for know correct answer for further learning; 0193 detail adding data (all factors mentioned above) to database (storage unit) for relearning the learner, and learner is updated (step S2405) where database may be external database) . Claim 10: SUZUKI et al teach The information processing apparatus according to claim 7, wherein if the first basis information or the second basis information is correct, the storage unit stores the first basis information or the second basis information that is selected by the user via the GUI, as learning data of the plurality of recognizers and the classifier (0171 detail receiving correction instruction from doctor, through GUI, where such correction is data for database for further learning/relearning of paragraph 0193) . Claim 11: SUZUKI et al teach: The information processing apparatus according to claim 7, wherein if the first behavior information and the second basis information are correct, the storage unit stores new basis information input by the user via the GUI, as learning data of the plurality of recognizers and the classifier (0171 detail receiving correction instruction from doctor, through GUI, where such correction is data for database for further learning/relearning of paragraph 0193) . Allowable Subject Matter Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. At the time of examination unable to find prior art regarding teaching of “wherein if the first behavior information and the second behavior information are incorrect, the storage unit stores third behavior information as learning data of the plurality of recognizers and the classifier, the third behavior information being different from the first behavior information and the second behavior information that are input by the user via the GUI.” Claim 12 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. At the time of examination unable to find prior art regarding teaching of “wherein if the first basis information and the second basis information are incorrect, the storage unit stores third basis information as learning data of the plurality of recognizers and the classifier, the third basis information being different from the first basis information and the second basis information that are input by the user via the GUI.” Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. FATHOLLAHI GHEZELGHIEH et al (US 2023/0177703) teaches TRACKING MULTIPLE SURGICAL TOOLS IN A SURGICAL VIDEO - apply deep-learning tool detection model 114 on the video frame, and generate one or more detected objects 124 (in the form of localized and classified bounding boxes) as outputs. In some embodiments, deep-learning tool detection model 114 is implemented based on a Faster-RCNN (regions convolutional neural network) architecture. Specifically, such a Faster-RCNN-based detection model has been trained to localize each surgical tool within each frame of the video, such as in each preprocessed video frame 122. In some embodiments, localizing a given surgical tool within the video frame includes generating a bounding box around the end effector portion of the detected surgical tool (0039). Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSUNG-YIN TSAI whose telephone number is (571)270-1671. The examiner can normally be reached 7am-4pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at (571) 272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /TSUNG YIN TSAI/Primary Examiner, Art Unit 2656 Application/Control Number: 18/838,845 Page 2 Art Unit: 2656 Application/Control Number: 18/838,845 Page 3 Art Unit: 2656 Application/Control Number: 18/838,845 Page 4 Art Unit: 2656 Application/Control Number: 18/838,845 Page 5 Art Unit: 2656 Application/Control Number: 18/838,845 Page 6 Art Unit: 2656 Application/Control Number: 18/838,845 Page 7 Art Unit: 2656 Application/Control Number: 18/838,845 Page 8 Art Unit: 2656 Application/Control Number: 18/838,845 Page 9 Art Unit: 2656 Application/Control Number: 18/838,845 Page 10 Art Unit: 2656 Application/Control Number: 18/838,845 Page 11 Art Unit: 2656 Application/Control Number: 18/838,845 Page 12 Art Unit: 2656 Application/Control Number: 18/838,845 Page 13 Art Unit: 2656 Application/Control Number: 18/838,845 Page 14 Art Unit: 2656
Prosecution Insights