Prosecution Insights
Last updated: November 11, 2026
Application No. 18/622,860

SYSTEM OF REAL-TIME DISPLAYING PROMPT IN SYNCHRONOUS DISPLAYED SURGICAL OPERATION VIDEO AND METHOD THEREOF

Final Rejection §103
Filed
Mar 29, 2024
Priority
Mar 31, 2023 — TW 112112567
Examiner
BOICE, JAMES EDWARD
Art Unit
3795
Tech Center
3700 — Mechanical Engineering & Manufacturing
Assignee
Smart Surgery
OA Round
2 (Final)
75%
Grant Probability
Favorable
3-4
OA Rounds
1m
Est. Remaining
84%
With Interview

Examiner Intelligence

Grants 75% — above average
75%
Career Allowance Rate
104 granted / 138 resolved
+5.4% vs TC avg
Moderate +8% lift
Without
With
+8.2%
Interview Lift
resolved cases with interview
Typical timeline
2y 9m
Avg Prosecution
33 currently pending
Career history
197
Total Applications
across all art units

Statute-Specific Performance

§101
0.4%
-39.6% vs TC avg
§103
57.7%
+17.7% vs TC avg
§102
21.5%
-18.5% vs TC avg
§112
17.5%
-22.5% vs TC avg
Black line = Tech Center average estimate • Based on career data from 138 resolved cases

Office Action

§103
Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This Office Action is in response to the amendments dated June 30, 2026. Claims 1-2, 5-8, and 10-15 are pending. Claim Interpretation The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are found in Claim 7, and include: an image processing module; a message obtaining module; a target determining module; a position determining module; and a label generating module. Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. More specifically: an image processing module is interpreted as image processing module 160 in FIG. 1B of the present patent application; a message obtaining module is interpreted as message obtaining module 120 in FIG. 1B of the present patent application; a target determining module is interpreted as target determining module 130 in FIG. 1B of the present patent application; a position determining module is interpreted as position determining module 140 in FIG. 1B of the present patent application; and a label generating module is interpreted as label generating module 150 in FIG. 1B of the present patent application. For purposes of examination, these modules are interpreted as executable software and/or executable firmware and/or hardware subcomponents of the processing module 101 shown in FIG. 1B of the present patent application. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The present rejection(s) reference specific passages from cited prior art. However, Applicant is advised that the rejections are based on the entirety of each cited prior art. That is, each cited prior art reference “must be considered in its entirety”. Therefore, Applicant is advised to review all portions of the cited prior art if traversing a rejection based on the cited prior art. Claims 1-2, 5, 7-8, 10, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Shioda et al. (US PGPUB 2017/0289528 – “Shioda”) in view of Amanatullah (US PGPUB 2019/0231433 – “Amanatullah”) and Chiang et al. (US PGPUB 2011/0199370 – “Chiang). Regarding Claim 1, Shioda discloses: A method of real-time displaying a synchronous displayed surgical operation video (Shioda FIG. 2, combination processing section 25; Shioda paragraph [0072], “process by the combination processing section 25, a process in the case of enabling the user to observe a 3D on the basis of glasses-free 3D picture technology”) applicable to a device (Shioda FIG. 2, medical stereoscopic observation device 9), and the method comprising: during a surgery, synchronously capturing two two-dimensional (2D) surgical operation videos from different viewing angles (Shioda FIG. 2, first imaging section 91a and second imaging section 91b; Shioda paragraph [0064], “first imaging section 91a and the second imaging section 91b capture an imaging target from mutually different viewpoints”), by the device (Shioda FIG. 2, medical stereoscopic observation device 9); generating a naked eye three-dimensional (3D) video corresponding to a 3D display based on the two 2D surgical operation videos in real time (Shioda FIG. 2, combination processing section 25; Shioda paragraph [0072], “combination processing section 25, a process in the case of enabling the user to observe a 3D on the basis of glasses-free 3D picture technology”), by the device; using the 3D display to synchronously project the two 2D surgical operation videos to left and right eyes of a viewer based on the naked eye 3D video, respectively, to make the viewer watch a 3D surgical operation video (Shioda FIG. 1, display device 550; Shioda paragraph [0072], “combination processing section 25 generates a desired multiview image presented on the basis of the computed results of the parallax value by the viewpoint images observed by the user's left and right eyes, respectively, and causes the generated multiview image to be displayed on a certain display device (for example, the display device 550 illustrated in FIG. 1)), by the device; Shioda does not explicitly disclose: receiving an instructional voice or detecting an instructional gesture within a surgical range to obtain an instruction message, or generating the instruction message based on an instructional operation for the naked eye 3D video displayed on a screen, by the device. analyzing a content of the instructional voice, or determining a position of the instructional gesture or the instructional operation in the naked eye 3D video, to determine a target part related to the instruction message, by the device; determining a label position of the target part in each of the two 2D surgical operation videos based on feature data of the target part, by the device; and generating a prompt corresponding to the target part in the two 2D surgical operation videos based on the label positions, to make the prompt be displayed in the 3D surgical operation video watched by the viewer, by the device. Amanatullah is analogous art in the field of image/label generation that teaches: receiving an instructional voice or detecting an instructional gesture within a surgical range to obtain an instruction message, or generating the instruction message based on an instructional operation for the naked eye 3D video displayed on a screen (Amanatullah paragraph [0035], “the computer system can implement template matching techniques to match template tissue point clouds—labeled with one or more anatomical tissue labels—to tissue masses identified in the 3D point cloud and transfer anatomical tissue labels from matched template tissue point clouds to corresponding tissue masses in the 3D point cloud”), by the device. analyzing a content of the instructional voice, or determining a position of the instructional gesture or the instructional operation in the naked eye 3D video, to determine a target part related to the instruction message (Amanatullah paragraph [0035], “the computer system can implement template matching techniques to match template tissue point clouds—labeled with one or more anatomical tissue labels—to tissue masses identified in the 3D point cloud and transfer anatomical tissue labels from matched template tissue point clouds to corresponding tissue masses in the 3D point cloud”), by the device; determining a label position of the target part in each of the two 2D surgical operation videos based on feature data of the target part (Amanatullah paragraph [0050], “the computer system can: transform 2D optical scans captured by cameras within the operating room into a 3D surgical field image”; Examiner interprets Amanatullah’s teaching that the 3D images are generated from 2D optical scans as teaching that the labels are on each of the two 2D images/videos), by the device; and generating a prompt corresponding to the target part in the two 2D surgical operation videos based on the label positions, to make the prompt be displayed in the 3D surgical operation video watched by the viewer (Amanatullah FIG. 1A, 3D virtual patient model with structural labels; The term “prompt” is not specifically described or defined in the specification. Examiner interprets “prompt” as a label that prompts a user to know what is described in the label.), by the device. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Amanatullah’s image labeling with the method disclosed by Shioda. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a method for identifying not only where, but also what, objects in a video are (see Amanatullah paragraphs [0035] and [0051]), in order for the user to fully understand what they are looking at. As described above, Amanatullah discloses generating 3D images from 2D images (Amanatullah paragraph [0050], “the computer system can: transform 2D optical scans captured by cameras within the operating room into a 3D surgical field image”), which results in the labels from the 2D images carrying over to the 3D images (Amanatullah paragraph [0035], “the computer system can implement template matching techniques to match template tissue point clouds—labeled with one or more anatomical tissue labels—to tissue masses identified in the 3D point cloud and transfer anatomical tissue labels from matched template tissue point clouds to corresponding tissue masses in the 3D point cloud”). Examiner interprets this combination of teachings from Amanatullah as teaching making the prompt/label to be displayed in the 3D surgical operation videos as corresponding to the prompt/label generated for the 2D surgical operation videos. Nonetheless, in an effort to promote compact prosecution, Chiang et al. (US PGPUB 2011/0199370- “Chiang”) is further cited for teaching this feature as described in Chiang FIG. 3B and Chiang paragraph [0033], “in step 32200…configuring a plurality of 2D feature label points to the reference image data according to the feature information, for generating a plurality of 2D feature label points; in step 32300, generating a plurality of 3D feature label points according to the plurality of 2D feature label points”). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to utilize Chiang’s 2D-to-3D label process in the method taught by Shioda in view of Amanatullah. A person having ordinary skill in the art would be motivated use a known technique (Chiang) to improve a similar method in the same way (i.e., creating a 3D label in a process that similar to the method taught by Amanatullah as described above). Regarding Claim 2, Shioda in view of Amanatullah and Chiang teaches the features of Claim 1, as described above. Amanatullah further teaches displaying a color block label or a text description for indicating the target part (Amanatullah FIG. 1A, “UNRESECTED TISSUE OF INTEREST”) corresponding to the label positions in the two 2D surgical operation videos, by the device, wherein the target part comprises an organ, a tissue (Amanatullah FIG. 1A, “UNRESECTED TISSUE OF INTEREST”), or an instrument. Regarding Claim 5, Shioda in view of Amanatullah and Chiang teaches the features of Claim 1, as described above. Shioda further discloses using an image capturing device (Shioda FIG. 2, imaging unit 90) with dual camera lenses (Shioda first imaging section 91a and second imaging section 91b; Shioda paragraph [0063], “the optical system includes various types of lenses”) to capture the two 2D surgical operation videos (Shioda FIG. 1, surgical video microscope device 510; Shioda paragraph [0046], “FIG. 1 illustrates an example of a case for an applied example of using a medical stereoscopic observation device according to an embodiment of the present disclosure, in which a surgical video microscope device equipped with an arm is used as the medical stereoscopic observation device.”), by the device. Regarding Claim 7, Shioda discloses: A system of real-time displaying a surgical operation video (Shioda FIG. 2, medical stereoscopic observation device 9), applicable to a device or multiple devices connected to each other, and the system comprising, an image capturing module (Shioda FIG. 2, imaging unit 90), configured to synchronously capture two 2D surgical operation videos from different viewing angles during a surgery (Shioda FIG. 2, first imaging section 91a and second imaging section 91b; Shioda paragraph [0064], “first imaging section 91a and the second imaging section 91b capture an imaging target from mutually different viewpoints”); an image display module (Shioda FIG. 1, display device 550), comprising a 3D display (Shioda paragraph [0056], “a mechanism for enabling the user 520 to observe, as a 3D image, the images displayed on the display device 550 as the left-eye image and the right-eye image”); and a processing module (Shioda FIG. 15, information processing apparatus 900; Shioda paragraph [0194], “an information processing apparatus 900 constituting a medical stereoscopic observation device according to the present embodiment, such as the surgical video microscope device or the image processing device discussed earlier”), connected to the image capturing module (Shioda FIG. 2, imaging unit 90) and the image display module (Shioda FIG. 2, parallax image; Shioda FIG. 1, display device 550; Shioda paragraph [0071], “the combination processing section 25 generates a parallax image having a set parallax enabling the user to observe a 3D image”) and configured to execute the computer readable instructions to generate: an image processing module (Shioda FIG. 2, combination processing section 25), configured to generate a naked eye 3D video corresponding to the 3D display based on the two 2D surgical operation videos in real time, to make the 3D display synchronously project the two 2D surgical operation videos to the left and right eyes of a viewer based on the naked eye 3D video, respectively, to make the viewer watch a 3D surgical operation video (Shioda paragraph [0072], “combination processing section 25, a process in the case of enabling the user to observe a 3D on the basis of glasses-free 3D picture technology”). Shioda does not explicitly disclose: a message obtaining module, configured to receive an instructional voice or detects an instructional gesture within a surgical range to obtain an instruction message, or generate the instruction message based on an instructional operation for the naked eye 3D video display on a screen of the 3D display; a target determining module, configured to determine a target part related to the instruction message; a position determining module, configured to determine a label position of the target part in each of the two 2D surgical operation videos based on feature data of the target part; and a label generating module, configured to generate a prompt corresponding to the target part in the two 2D surgical operation videos based on the label positions, to make the prompt be displayed in the 3D surgical operation video watched by the viewer. Amanatullah is analogous art in the field of image/label generation that teaches: A system for displaying a prompt (Amanatullah FIG. 1A, labels on 3D virtual patient model generated by method S100; Amanatullah paragraph [0014], “a computer system can execute Blocks of the method S100 to access and transform scan data of a hard tissue of interest (e.g., bone) of a patient into a virtual patient model representing the hard tissue of interest prior to a surgical operation on the patient”); a message obtaining module (Amanatullah paragraph [0171], computer-executable component can be a processor but any dedicated hardware device can (alternatively or additionally) execute the instructions”), configured to receive an instructional voice or detects an instructional gesture within a surgical range to obtain an instruction message, or generate the instruction message based on an instructional operation for the naked eye 3D video display on a screen of the 3D display (Amanatullah FIG. 1A, 3D virtual patient model with structural labels; Amanatullah paragraph [0035], “write an anatomical tissue label to each distinct tissue mass in the 3D point cloud based on anatomical tissue labels manually entered or selected by the surgeon through the physician portal”); a target determining module (Amanatullah paragraph [0171], computer-executable component can be a processor but any dedicated hardware device can (alternatively or additionally) execute the instructions”), configured to determine a target part related to the instruction message (Amanatullah paragraph [0035], “the computer system can implement template matching techniques to match template tissue point clouds—labeled with one or more anatomical tissue labels—to tissue masses identified in the 3D point cloud and transfer anatomical tissue labels from matched template tissue point clouds to corresponding tissue masses in the 3D point cloud”); a position determining module (Amanatullah paragraph [0171], computer-executable component can be a processor but any dedicated hardware device can (alternatively or additionally) execute the instructions”), configured to determine a label position of the target part in each of the two 2D surgical operation videos based on feature data of the target part (Amanatullah paragraph [0050], “the computer system can: transform 2D optical scans captured by cameras within the operating room into a 3D surgical field image”; Examiner interprets Amanatullah’s teaching that the 3D images are generated from 2D optical scans as teaching that the labels are on each of the two 2D images/videos); and a label generating module (Amanatullah paragraph [0171], computer-executable component can be a processor but any dedicated hardware device can (alternatively or additionally) execute the instructions”), configured to generate a prompt corresponding to the target part in the two 2D surgical operation videos based on the label positions, to make the prompt be displayed in the 3D surgical operation video watched by the viewer (Amanatullah FIG. 1A, 3D virtual patient model with structural labels). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Amanatullah’s computer modules, which are created by Amanatullah’s computer/processor, with the device disclosed by Shioda. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a device that identifies not only where, but also what, objects in a video are (see Amanatullah paragraphs [0035] and [0051]), in order for the user to fully understand what they are looking at. As described above, Amanatullah discloses generating 3D images from 2D images (Amanatullah paragraph [0050], “the computer system can: transform 2D optical scans captured by cameras within the operating room into a 3D surgical field image”), which results in the labels from the 2D images carrying over to the 3D images (Amanatullah paragraph [0035], “the computer system can implement template matching techniques to match template tissue point clouds—labeled with one or more anatomical tissue labels—to tissue masses identified in the 3D point cloud and transfer anatomical tissue labels from matched template tissue point clouds to corresponding tissue masses in the 3D point cloud”). Examiner interprets this combination of teachings from Amanatullah as teaching making the prompt/label to be displayed in the 3D surgical operation videos as corresponding to the prompt/label generated for the 2D surgical operation videos. Nonetheless, in an effort to promote compact prosecution, Chiang et al. (US PGPUB 2011/0199370- “Chiang”) is further cited for teaching this feature as described in Chiang FIG. 3B and paragraph [0033], “in step 32200…configuring a plurality of 2D feature label points to the reference image data according to the feature information, for generating a plurality of 2D feature label points; in step 32300, generating a plurality of 3D feature label points according to the plurality of 2D feature label points”). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to utilize Chiang’s 2D-to-3D label process in the system taught by Shioda in view of Amanatullah. A person having ordinary skill in the art would be motivated use a known technique (Chiang) to improve a similar method in the same way (i.e., creating a 3D label in a process that similar to the method taught by Amanatullah as described above). Regarding Claim 8, Shioda in view of Amanatullah and Chiang teaches the features of Claim 7, as described above. Amanatullah further teaches wherein the label generating module displays a color block label or a text description for indicating the target part (Amanatullah FIG. 1A, “UNRESECTED TISSUE OF INTEREST”) corresponding to the label positions in the two 2D surgical operation videos, to generate the prompt, wherein the target part comprises an organ, a tissue (Amanatullah FIG. 1A, “UNRESECTED TISSUE OF INTEREST”), or an instrument. Regarding Claim 10, Shioda in view of Amanatullah and Chiang teaches the features of Claim 7, as described above. Shioda further discloses wherein the image capturing module comprises an image capturing device (Shioda FIG. 2, imaging unit 90) with dual camera lenses (Shioda first imaging section 91a and second imaging section 91b; Shioda paragraph [0063], “the optical system includes various types of lenses”) to capture the two 2D surgical operation videos (Shioda FIG. 1, surgical video microscope device 510; Shioda paragraph [0046], “FIG. 1 illustrates an example of a case for an applied example of using a medical stereoscopic observation device according to an embodiment of the present disclosure, in which a surgical video microscope device equipped with an arm is used as the medical stereoscopic observation device.”) Regarding Claim 13, Shioda in view of Amanatullah and Chiang teaches the features of Claim 1, as described above. Shioda further discloses wherein step of generating the naked eye 3D video by the device comprises: arranging vertical pixels or pixel columns of synchronous frames in the 2D surgical operation video in sequential order to generate the naked eye 3D video, or extracting relative positions or capturing angles of an image capturing devices capturing the 2D surgical operation videos and combining the 2D surgical operation videos in sequential order to generate the naked eye 3D video (Shioda FIG. 3 first imaging section 11a and second imaging section 11b; Shioda paragraph [0005], “by causing the viewpoint images captured by mutually different imaging sections to be observed by the left and right eyes, it becomes possible to enable a user to observe an image of the affected area as a stereoscopic 3D image based on the parallax between the viewpoint images”. See also Shioda paragraph [0055]). Regarding Claim 15, Shioda in view of Amanatullah and Chiang teaches the features of Claim 7, as described above. Shioda further discloses wherein the image processing module arranges vertical pixels or pixel columns of synchronous frames in the 2D surgical operation video in sequential order to generate the naked eye 3D video, or extracts relative positions or capturing angles of an image capturing devices capturing the 2D surgical operation videos and combines the 2D surgical operation videos in sequential order to generate the naked eye 3D video (Shioda FIG. 3 first imaging section 11a and second imaging section 11b; Shioda paragraph [0005], “by causing the viewpoint images captured by mutually different imaging sections to be observed by the left and right eyes, it becomes possible to enable a user to observe an image of the affected area as a stereoscopic 3D image based on the parallax between the viewpoint images”. See also Shioda paragraph [0055]). Claims 6 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Shioda et al. (US PGPUB 2017/0289528 – “Shioda”) in view of Amanatullah (US PGPUB 2019/0231433 – “Amanatullah”), Chiang et al. (US PGPUB 2011/0199370 – “Chiang), and Russell (US PGPUB 2020/0021796 – “Russell”). Regarding Claim 6, Shioda in view of Amanatullah and Chiang teaches the features of Claim 1, as described above. Shioda in view of Amanatullah and Chiang does not explicitly teach: detecting dual-eye dynamics and a head movement of the viewer to determine a watch sight of the viewer, and adjusting viewing angles of the 3D display projecting the two 2D surgical operation videos based on the watch sight, by the device. Russell is analogous art in the field of image generation that teaches: detecting dual-eye dynamics and a head movement of the viewer to determine a watch sight of the viewer, and adjusting viewing angles of the 3D display projecting the two 2D surgical operation videos based on the watch sight, by the device (Russell FIG. 2, tracking system 230; Russell paragraph [0072], “tracking system 230 may track a location of the user (or location of the eyes or head of the user) with respect to a display device 202, for example, may be used to configure masks for a left and a right eye of the user to display images with proper depth and parallax.”). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Russell’s user tracking system with the method taught by Shioda in view of Amanatullah and Chiang. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a method that allows a user to view a 3D display without glasses and with little distortion of the 3D image (see Russell paragraph [0039]). Regarding Claim 11, Shioda in view of Amanatullah and Chiang teaches the features of Claim 7, as described above. Shioda in view of Amanatullah and Chiang does not explicitly teach: wherein the processing module comprises a sight detecting module and a viewing angle adjusting module, the sight detecting module detects dual-eye dynamics and a head movement of the viewer to determine a watch sight of the viewer, and the viewing angle adjusting module adjusts viewing angles of the 3D display projecting the two 2D surgical operation videos based on the watch sight. Russell is analogous art in the field of image generation that teaches: wherein the processing module comprises a sight detecting module (Amanatullah paragraph [0171], computer-executable component can be a processor but any dedicated hardware device can (alternatively or additionally) execute the instructions”) and a viewing angle adjusting module (Amanatullah paragraph [0171], computer-executable component can be a processor but any dedicated hardware device can (alternatively or additionally) execute the instructions”), the sight detecting module detects dual-eye dynamics and a head movement of the viewer to determine a watch sight of the viewer, and the viewing angle adjusting module adjusts viewing angles of the 3D display projecting the two 2D surgical operation videos based on the watch sight (Russell FIG. 2, tracking system 230; Russell paragraph [0072], “tracking system 230 may track a location of the user (or location of the eyes or head of the user) with respect to a display device 202, for example, may be used to configure masks for a left and a right eye of the user to display images with proper depth and parallax.”). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Russell’s user tracking system with the device taught by Shioda in view of Amanatullah and Chiang. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a device that allows a user to view a 3D display without glasses and with little distortion of the 3D image (see Russell paragraph [0039]). Claims 12 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Shioda et al. (US PGPUB 2017/0289528 – “Shioda”) in view of Amanatullah (US PGPUB 2019/0231433 – “Amanatullah”), Chiang et al. (US PGPUB 2011/0199370 – “Chiang), and Yokota (US PGPUB 2014/0071239 – “Yokota”). Regarding Claim 12, Shioda in view of Amanatullah and Chiang teaches the features of Claim 1, as described above. Shioda in view of Amanatullah and Chiang does not explicitly teach wherein step of determining the target part related to the instruction message by the device comprises: determining whether the instructional voice includes a speech signal having high- enough matching degree or similarity with an organ name, a tissue name, or an instrument name related to the surgery, or performing a speech recognition on the instructional voice to obtain an instruction content of the instructional voice, and when determining whether the instruction content has text of an organ name, a tissue name or an instrument name related to the surgery, using the organ or tissue or instrument in the instruction content, as the target part, by the device; or performing a feature analysis on the instructional gesture, and determining the related target part based on the analyzed gesture feature, or determining the organ or tissue or instrument pointed or circled by the instructional gesture based on the gesture feature of the instructional gesture, and using the organ or tissue or instrument pointed or circled by the instructional gesture as the target part, by the device. Yokota is analogous art in the field of image feature recognition that teaches determining whether the instructional voice includes a speech signal having high-enough matching degree or similarity with an organ name, a tissue name, or an instrument name related to the surgery, or performing a speech recognition on the instructional voice to obtain an instruction content of the instructional voice, and when determining whether the instruction content has text of an organ name, a tissue name or an instrument name related to the surgery, using the organ or tissue or instrument in the instruction content, as the target part, by the device; or performing a feature analysis on the instructional gesture, and determining the related target part based on the analyzed gesture feature, or determining the organ or tissue or instrument pointed or circled by the instructional gesture based on the gesture feature of the instructional gesture, and using the organ or tissue or instrument pointed or circled by the instructional gesture as the target part (Yokota FIG. 5, target T identified by circle created by user; Yokota paragraph [0077], “cursor C may have a shape enabling the user to easily designate coordinates or a region of the specimen…the cursor C may be used to designate an area of the target region with a shape of a circle mark”), by the device. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Yokota’s method of a user using a circle to identify a target area with the method taught by Shioda in view of Amanatullah. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a method that allows a user to identify a particular target during an operation in real time, in order to identify the boundaries used when identifying/labeling the target using edge detection (see Amanatullah paragraph [0035]). Regarding Claim 14, Shioda in view of Amanatullah and Chiang teaches the features of Claim 7, as described above. Shioda in view of Amanatullah and Chiang does not explicitly teach wherein the target determining module determines whether the instructional voice includes a speech signal having high-enough matching degree or similarity with an organ name, a tissue name, or an instrument name related to the surgery, or performs a speech recognition on the instructional voice to obtain an instruction content of the instructional voice, and when determining whether the instruction content has text of an organ name, a tissue name or an instrument name related to the surgery, uses the organ or tissue or instrument in the instruction content, as the target part, or performs a feature analysis on the instructional gesture, and determines the related target part based on the analyzed gesture feature, or determines the organ or tissue or instrument pointed or circled by the instructional gesture based on the gesture feature of the instructional gesture, and uses the organ or tissue or instrument pointed or circled by the instructional gesture as the target part. Yokota is analogous art in the field of image processing that teaches wherein the target determining module determines whether the instructional voice includes a speech signal having high-enough matching degree or similarity with an organ name, a tissue name, or an instrument name related to the surgery, or performs a speech recognition on the instructional voice to obtain an instruction content of the instructional voice, and when determining whether the instruction content has text of an organ name, a tissue name or an instrument name related to the surgery, uses the organ or tissue or instrument in the instruction content, as the target part, or performs a feature analysis on the instructional gesture, and determines the related target part based on the analyzed gesture feature, or determines the organ or tissue or instrument pointed or circled by the instructional gesture based on the gesture feature of the instructional gesture, and uses the organ or tissue or instrument pointed or circled by the instructional gesture as the target part (Yokota FIG. 5, target T identified by circle created by user; Yokota paragraph [0077], “cursor C may have a shape enabling the user to easily designate coordinates or a region of the specimen…the cursor C may be used to designate an area of the target region with a shape of a circle mark”). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Yokota’s method of a user using a circle to identify a target area with the device taught by Shioda in view of Amanatullah and Chiang. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a method that allows a user to identify a particular target during an operation in real time, in order to identify the boundaries used when identifying/labeling the target using edge detection (see Amanatullah paragraph [0035]). Response to Arguments Applicant's response filed June 30, 2026 does not traverse/dispute the interpretation of Claim 7 under 35 U.S.C. 112(f), as presented in the April 1, 2026 non-final Office Action. As such, the features found in Claim 7 as described above and in the April 1, 2026, non-final Office Action continue to be interpreted under 35 U.S.C. 112(f). Applicant's arguments, filed on June 30, 2026, with regard to the rejections of Claims 1 and 7 under 35 U.S.C. 103 have been fully considered but they are not persuasive. The arguments found on pages 9-11 and later on pages 12-13 assert that target labels are assigned to targets by the user during a surgical procedure, not beforehand. However, Claims 1 and 7, particularly under their broadest reasonable interpretation, do not claim this feature. The only feature claimed to occur during a surgery is the capture of 2D surgical operation videos. That is Claim 1 claims “during a surgery, synchronously capturing two two-dimensional (2D) surgical operation videos from different viewing angles, by the device”, and Claim 7 claims “an image capturing module, configured to synchronously capture two 2D surgical operation videos from different viewing angles during a surgery” (Claim 7). If Applicant intended for all steps in Claims 1 and 7 to be performed during a surgery, the Examiner refers Applicant to previously-uncited Kelley (US PGPUB 2022/0031405 – “Kelley”) in the conclusion below. On pages 11-12, Applicant asserts that Amanatullah fails to teach applying a label/prompt from two 2D videos onto a 3D video. Examiner believes that Amanatullah discloses generating 3D images from 2D images (Amanatullah paragraph [0050], “the computer system can: transform 2D optical scans captured by cameras within the operating room into a 3D surgical field image”), which results in the labels from the 2D images carrying over to the 3D images (Amanatullah paragraph [0035], “the computer system can implement template matching techniques to match template tissue point clouds—labeled with one or more anatomical tissue labels—to tissue masses identified in the 3D point cloud and transfer anatomical tissue labels from matched template tissue point clouds to corresponding tissue masses in the 3D point cloud”). Nonetheless, in an effort to promote compact prosecution, the present rejection of Claims 1 and 7 under 35 U.S.C. 103 further references the teachings of analogous art Chiang et al. (US PGPUB 2011/0199370- “Chiang”), as described above. Finally, on page 13, Applicant asserts that it would not be obvious to combine Shioda’s method of generating a low-latency stereoscopic display of live surgical videos with Amanatullah’s model reconstruction and label registration process, since Amanatullah requires the consumption of “enormous computation resources”, thus slowing down the method disclosed by Shioda. First, Examiner is unable to find in the record any explanation of what constitutes “enormous computation resources”. Second, even if additional computational resources were required, slowing down Shioda’s stereoscopic display would not constitute a modification that would render Shioda unsatisfactory for its intended purpose, particularly since this computation time is not defined or obvious from the reading of Amanatullah. See MPEP 2143.01(V), “[a] given course of action often has simultaneous advantages and disadvantages, and this does not necessarily obviate motivation to combine", Medichem, S.A. v. Rolabo, S.L., 437 F.3d 1157, 1165, 77 USPQ2d 1865, 1870 (Fed. Cir. 2006) (citation omitted)”). Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure includes, but is not limited to: Kelley (US PGPUB 2022/0031405 – “Kelley”) teaches in Kelley FIG. 4 a method in which a user labels a target, as described in Kelley paragraph [0052] (“In operation 404…the processor 116 receives user input (e.g., clinician input) from a user interface of an application that is rendering the image that indicates whether the label of the object of interest is accurate. If not…the processor 116 may take other action by requesting feedback from the user as to an accurate label for the object of interest in the candidate image. The feedback may be audio in nature (e.g., a clinician's voice) and/or tactile in nature (e.g., input on a keyboard or mouse).”). Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIM BOICE whose telephone number is (571)272-6565. The examiner can normally be reached Monday-Friday 9:00am - 5:00pm Eastern. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571)272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. JIM BOICE Examiner Art Unit 3795 /JAMES EDWARD BOICE/Examiner, Art Unit 3795 /ANHTUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795 8/30/26
Read full office action

Prosecution Timeline

Mar 29, 2024
Application Filed
Apr 01, 2026
Non-Final Rejection mailed — §103
Jun 30, 2026
Response Filed
Sep 02, 2026
Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12778385
METHOD AND SYSTEM FOR SELF-CHECKING OF AN ENDOSCOPE REPROCESSING DEVICE
3y 1m to grant Granted Nov 03, 2026
Patent 12778396
ENDOSCOPE COMPRISING A SUCTION VALVE
2y 2m to grant Granted Nov 03, 2026
Patent 12721610
ELONGATE FLEXIBLE SYSTEMS WITH ARTICULATION JOINT ASSEMBLY
3y 10m to grant Granted Sep 01, 2026
Patent 12702402
ELECTRIC OR MANUAL GEARED SUTURING DEVICE
2y 8m to grant Granted Aug 11, 2026
Patent 12670549
VISUALIZATION ADJUSTMENTS FOR INSTRUMENT ROLL
3y 1m to grant Granted Jun 30, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

3-4
Expected OA Rounds
75%
Grant Probability
84%
With Interview (+8.2%)
2y 9m (~1m remaining)
Median Time to Grant
Moderate
PTA Risk
Based on 138 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month