Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-11 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Avisar et al., US PGPUB 20180322806 hereinafter referenced as Avisar in view of Cheong et al., US PGPUB 20140232727 hereinafter referenced as Cheong.
As to claim 1, Avisar discloses a method of an electronic device, the method comprising: displaying a first image (e.g., anatomy display 18, fig. 1);
receiving a user input forming a drawing on the first image ([0005] a user tool generator for generating a tool model of a user tool for dynamically interacting with the dynamic image of tissues via manipulations provided by a user input for display on the display);
identifying a shape of the drawing and a position at which the drawing is formed ([0107] “Tissue Painting”—the developed algorithm and software tool provides the user an interface to draw any geometric shape or free hand drawing shape in 2- or 3-dimensions (e.g., line, circle, clinic, ball etc.));
identifying an object, positioned at the identified position, included in the first image and space information related to a space represented by the first image on the first image corresponding to the shape of the gesture drawing input on the first image ([0036] The system 1 performs a segmentation process and identified the Entities of the organ, Entities are vessels, tissue, tumor, and so on to create the simulated image model 18 shown to the surgeon on the display of the device);
determining, based on the space information and the identified object, lighting information indicating at least one parameter for generating a virtual light having the shape of the drawing ([0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.); and
generating a second image by applying the virtual lighting to the identified object included in the first image ([0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
Avisar does not specifically discloses space information related to a space represented by the first image.
However, in the same endeavor, Cheong discloses space information related to a space represented by the first image ([0074] The attributes of the writing data indicate the position, the size, and the feature of the writing data to generate. The type of the writing data indicates the shape of the writing data to generate).
Therefore, it would have been obvious to one of ordinary skill in the art to modify the disclosure of Avisar to further include Huang’s position identification method, in order to enhance the image with intention of increasing image smoothness.
As to claim 11, Avisar discloses a n electronic device comprising: a touch display (e.g., anatomy display 18, fig. 1);
a memory comprising an instruction (e.g., databases 3A-3n, fig. 1A); and
at least one processor comprising processing circuitry, wherein at least one processor is configured to execute the instruction ([0063] The typical system's computer is a PC with a multiple core (multiple processors)), and
at least one processor, individually and/or collectively, is configured to cause the electronic device to: display a first image through the display; receive a user input forming a drawing on the first image ([0005] a user tool generator for generating a tool model of a user tool for dynamically interacting with the dynamic image of tissues via manipulations provided by a user input for display on the display);
identify a shape of the drawing and a position at which the gesture drawing is formed ([0107] “Tissue Painting”—the developed algorithm and software tool provides the user an interface to draw any geometric shape or free hand drawing shape in 2- or 3-dimensions (e.g., line, circle, clinic, ball etc.));
identify an object, positioned at the identified position, included in the first image and space information related to a space on represented by the first image ([0036] The system 1 performs a segmentation process and identified the Entities of the organ, Entities are vessels, tissue, tumor, and so on to create the simulated image model 18 shown to the surgeon on the display of the device);
determine, based on the space information and the identified object, lighting information indicating at least one parameter for generating a virtual light having the shape of the drawing ([0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.); and
generate a second image by applying the virtual light to the identified object included in the first image ([0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
Avisar does not specifically disclose space information related to a space represented by the first image.
However, in the same endeavor, Cheong discloses space information related to a space represented by the first image ([0074] The attributes of the writing data indicate the position, the size, and the feature of the writing data to generate. The type of the writing data indicates the shape of the writing data to generate).
Therefore, it would have been obvious to one of ordinary skill in the art to modify the disclosure of Avisar to further include Huang’s position identification method, in order to enhance the image with intention of increasing image smoothness.
2. (Cancelled)
As to claim 3, the combination of Avisar and Cheong discloses the method of claim 1. The combination further discloses the lighting information is applied, based on a spatial position relationship between the identified object and the virtual lighting, to the identified object, and wherein the spatial position relationship is identified based on image segmentation or depth information of the first image (Avisar, [0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 4, the combination of Avisar and Cheong discloses the method of claim 1. The combination further discloses displaying the lighting information on the first image through a display; and receiving an input of selection of the displayed lighting information (Avisar, [0112]) The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 5, the combination of Avisar and Cheong discloses the method of claim 4. The combination further discloses the lighting information comprises two or more pieces of lighting selection information (Avisar, [0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 6, the combination of Avisar and Cheong discloses the method of claim 5. The combination further discloses the input of selection comprises an input of selection of turning on or off the two or more pieces of lighting selection information, a detailed characteristic selection input, or a detailed characteristic adjustment input (Avisar, [0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 7, the combination of Avisar and Cheong discloses the method of claim 4. The combination further discloses identifying, based on the space information, two or more pieces of lighting information of the first image; displaying the two or more pieces of lighting information on the first image through the display; and receiving an input of selection of the two or more pieces of lighting information (Avisar, [0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
As to claim 8, the combination of Avisar and Cheong discloses the method of claim 1. The combination further discloses the space information comprises identification information of a space corresponding to the first image, style information, and physical space information (Avisar, [0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
As to claim 9, the combination of Avisar and Cheong discloses the method of claim 8. The combination further discloses the lighting information comprises type information, shape information, size information, or characteristic information of the virtual lighting, and wherein one or more of the type information, the shape information, the size information, or the characteristic information of the virtual lighting are determined based on one or more of the identification information of the space, the style information, or the physical space information (Avisar, [0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
As to claim 10, the combination of Avisar and Cheong discloses the method of claim 9. The combination further discloses providing, through a display, a menu for selecting or adjusting one or more of the type information, the shape information, the size information, or the characteristic information of the light (Avisar, [0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
12. (Cancelled)
As to claim 13, the combination of Avisar and Cheong discloses the electronic device of claim 11. The combination further discloses the at least one processor, individually and/or collectively, is configured to cause the electronic device to: apply, based on a spatial position relationship between the identified object and the virtual lighting, the virtual lighting to the identified object; and identify, based on image segmentation or depth information of the first image, the spatial position relationship (Avisar, [0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 14, the combination of Avisar and Cheong discloses the electronic device of claim 11. The combination further discloses the at least one processor, individually and/or collectively, is configured to cause the electronic device to: display the lighting information on the first image through the display; and receive an input of selection of the displayed space lighting information (Avisar, [0112]) The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 15, the combination of Avisar and Cheong discloses the electronic device of claim 14. The combination further discloses the lighting information comprises one or more pieces of lighting selection information (Avisar, [0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 16, the combination of Avisar and Cheong discloses the electronic device of claim 15. The combination further discloses the input of selection comprises an input of selection of turning on or off the two or more pieces of lighting selection information, a detailed characteristic selection input, or a detailed characteristic adjustment input (Avisar, [0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
As to claim 17, the combination of Avisar and Cheong discloses the electronic device of claim 14. The combination further discloses the at least one processor, individually and/or collectively, is configured to cause the electronic device to: identify, based on the space information, two or more images pieces of lighting information of the first image; display the two or more pieces of lighting information on the first image through the display; and receive an input of selection of the two or more pieces of lighting information (Avisar, [0112] Furthermore, the characteristics of shading and shadowing of the new created structure can be modified by tuning the characteristics of the virtual light sources. The virtual light sources characteristics includes: spherical location in space, color of the light, strength of the light, the aspect ratio, the geometric shape of the virtual source etc.).
As to claim 18, the combination of Avisar and Cheong discloses the electronic device of claim 11. The combination further discloses the space information comprises identification information of a space corresponding to the first image, style information, and physical space information (Avisar, [0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
As to claim 19, the combination of Avisar and Cheong discloses the electronic device of claim 18. The combination further discloses the lighting information comprises type information, shape information, size information, or characteristic information of the virtual lighting, and wherein the at least one processor, individually and/or collectively, is configured to cause the electronic device to determine, based on one or more of the identification of the space, the style information, or the physical space information, one or more of the type information, the shape information, the size information, or the characteristic information of the virtual lighting (Avisar, [0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
As to claim 20, the combination of Avisar and Cheong discloses the electronic device of claim 19. The combination further discloses the at least one processor, individually and/or collectively, is configured to cause the electronic device to provide, through the display, a menu for selecting or adjusting one or more of the type information, the shape information, the size information, or the characteristic information of the light (Avisar, [0016] adjusting, using a user input to the computer system, the dynamic image of tissues displayed on the display by adding or modifying features of the tissues for display to compensate for anatomical structures that are in the actual biological tissue of the particular patient but are missing from the dynamic image of tissues originally displayed such that the dynamic image of tissues displayed are subsequently displayed on the display with the added or modified features).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Pastrana et al., US PGPUB 20230100610 discloses a computer system displays a first and second user interface object in a three-dimensional environment. The first and second user interface objects have a first and second spatial relationship to a first and second anchor position corresponding to a location of a user's hand in a physical environment, respectively. While displaying the first and second user interface objects in the three-dimensional environment, the computer system detects movement of the user's hand in the physical environment, corresponding to a translational movement and a rotational movement of the user's hand relative to a viewpoint, and in response, translates the first and second user interface objects relative to the viewpoint in accordance with the translational movement of the user's hand, and rotates the first user interface object relative to the viewpoint in accordance with the rotational movement of the user's hand without rotating the second user interface object.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAHLU OKEBATO whose telephone number is (571)270-3375. The examiner can normally be reached Mon - Fri 8:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, WILLIAM BODDIE can be reached at 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAHLU OKEBATO/Primary Examiner, Art Unit 2625 8/21/2026
Prosecution Insights