DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Arguments
This is in response to applicant’s amendment/response filed on 11/10/2025 which have been entered and made of record.
Applicant’s arguments/amendments regarding claim rejections under 35 U.S.C. 112(b) rejections have been fully considered and are persuasive. 35 U.S.C. 112(b) rejections for claims 1, 2, 9-12, 14, 17, 22, and 23 have been overcome and are withdrawn.
Applicant’s arguments regarding claim rejections under 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant argues Ohyu teaches the determination of the augmentation data based on the input display data (see Office Action, p. 8) and alleges that Kim teaches the positioning and the generation of the augmentation position. See Office Action, p. 8-9. However, as discussed with the Examiner during the interview, Kim's determination of the augmentation position is based on predefined locations for the information (e.g., a predefined box to place the extracted diagnosis text from a medical record). In particular, the Office Action relics on portions of Kim related to Figs, 15-17, which teach use of multiple windows pulling content from multiple places but not based on the image data itself See Office Action, p. 9. The specification of Kim states, that "[FIGS.] 16 and 17 are diagrams illustrating an example of a configuration of a graphical user interface (GUI) according to an operation of an image data viewer in a client terminal according to the present invention. An example of simultaneously displaying an image file together with medical information on one screen is shown." Kim, p. 20. The specification continues. that ''[t]he first display window 1100 displays at least one image frame according to the reproduction of the image file of the medical image data. [, .. ]The second display window 1200 displays text of the corresponding medical information according to the reproduction of the medical information extracted from the medical image data." Kim, p. 21(emphasis added). Figures 16 and I 7 are reproduced herein; from these is it clear that, while the final output of the Kim may include an image, the positioning of the non-image annotation is not based on the data from the image.
Examiner respectable disagrees Kim teaches a medical image data processing system. The system receives encoded medical data which includes medical image data. The medical data is then extracted/decoded and reproduced onto a screen (Page 4, Section: Tech-Solution Para. 1-3). The medical data received can be unmodified data directly from a device that captures the medical data and modified data by doctors like annotations or diagnosis (Page. 6, Para. 2-4). Medical Images can be displayed individually or together (Fig. 16 and Fig. 17). The medical images received by the system can be classified into groups (Page 9, Para. 6) and can specify a main image for a specific diagnosis (Page 3, Para. 4-6). A doctor can modify and move the medical images within the display to specific positions for diagnosis (Page 21, Para. 2-5). Kim teaches the medical data can be shared remotely to many doctors to reconfirm the diagnosis or to get other doctor’s opinions (Page 6, Para. 4). Because of this when the medical data is extracted/decoded from a server and sent to another doctor. The input display data that includes the medical image data, would be analyzed to recreate the augmented positions of the augmented data. Such as, displaying the medical image determined to be the main image that shows the diagnosis with the annotations of various doctors. Thus, Kim teaches determine, based on analysing the one or more images comprising one or more pixels of the input display data, at least one augmentation position for displaying the augmentation data at the least one display, wherein at least one of the augmentation data and the at least one augmentation position is determined based on analysing an informational content of the user interface data.
Regarding the remaining arguments applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-6, 8-13, 15-16, 20-21, 23-28, 30-31, and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Ohyu et al. U.S. Patent Application Publication 20210407655 A1 in view of Kim et al. WIPO International Application 2010120061 A2.
Regarding claim 1, Ohyu teaches a processing device (Medical Data Processing Apparatus 1, Para. 0056) for analysing and/or augmenting display data (Para. 0055), comprising (Fig. 1):
an input interface (Input Interface 13, Para. 0061) configured to receive input display data (Image Data, Para. 0083) from an image rendering device (CT Apparatus or MRI Apparatus, Para. 0060), wherein the input display data includes one or more images comprising one or more pixels (Image Data, Para. 0083)
an output interface (Output Function 23, Para. 0068) configured to transmit output display data (Patterns extracted by Extracting Function 22, Para. 0068) to at least one display (Display 14, Para. 0068), wherein the input display data (Image Data, Para. 0083) includes user interface data (Displaying data, Para. 0059) indicative of a graphical user interface (GUI, Para. 0059) displayable at the at least one display; (Display 14, Para. 0059)
and one or more processors (Processing Circuitry 11, Para. 0057) configured to:
determine, based on analysing (Examination of Image Data Belongs to Pattern, Para. 0083) the input display data (Image data, Para. 0083), augmentation data for augmenting (Display only Necessary Patterns, Para. 0083) the input display data; (Image data, Para. 0083)
However, Ohyu fails to explicitly teach:
determine, based on analysing the one or more images comprising one or more pixels of the input display data, at least one augmentation position for displaying the augmentation data at the least one display, wherein at least one of the augmentation data and the at least one augmentation position is determined based on analysing an informational content of the user interface data; and
generate the output display data based on supplementing the input display data with the determined augmentation data and the determined at least one augmentation position, such that the output display data is displayable at the least one display.
Ohyu and Kim are analogous to the claimed invention because both of them are in the same field of developing medical data processing apparatuses that are used to process medical image data.
Kim teaches:
determine, based on analysing the one or more images (Medical Images, Page 3, Para. 1-3) comprising one or more pixels of the input display data (Medical Information and Image Frame, Page 4, Para. 4), at least one augmentation position (Position of Image Frame, Page 4, Para. 4) for displaying the augmentation data at the least one display, wherein at least one of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) and the at least one augmentation position (Position of Image Frame, Page 4, Para. 4) is determined based on analysing an informational content (Based on Display Window Content, Page 4, Para. 4) of the user interface data (Graphical User Interface, Page 4, Para. 4); and (Fig. 15-17)
generate the output display data based on supplementing the input display data (Medical Data, Page 17, Para. 6) with the determined augmentation data (Outputting Selected Medical Images Based on Buttons, Page 17, Para. 6) and the determined at least one augmentation position (Position of Image Frame, Page 4, Para. 4), such that the output display data is displayable at the least one display. (Fig.17) (Depending on the display window different text corresponding to medical information and image frames are displayed at different positions or grouped together. Page 4, Para. 4 The user can then interact with the user interface in many ways to view the medical data through buttons and tabs. Page 17, Para. 6, Fig. 17)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus’s Graphical User Interface to incorporate Kim’s Graphical User Interface. Since, doing so would provide the benefit of following a common standard. The ability to manipulate data using a graphical user interface or determine where in the graphical user interface data is placed based on the data is a common standard in graphical user interfaces. Since, graphical user interfaces are built to be user-friendly and interactive. Augmenting the data and position is a frequent occurrence in user interfaces by various modalities (Clicking, Moving Windows, Search Bars, Inputting Data, and etc.)
Additionally, as discussed above, Kim teaches a medical image data processing system. The system receives encoded medical data which includes medical image data. The medical data is then extracted/decoded and reproduced onto a screen (Page 4, Section: Tech-Solution Para. 1-3). The medical data received can be unmodified data directly from a device that captures the medical data and modified data by doctors like annotations or diagnosis (Page. 6, Para. 2-4). Medical Images can be displayed individually or together (Fig. 16 and Fig. 17). The medical images received by the system can be classified into groups (Page 9, Para. 6) and can specify a main image for a specific diagnosis (Page 3, Para. 4-6). A doctor can modify and move the medical images within the display to specific positions for diagnosis (Page 21, Para. 2-5). Kim teaches the medical data can be shared remotely to many doctors to reconfirm the diagnosis or to get other doctor’s opinions (Page 6, Para. 4). Because of this when the medical data is extracted/decoded from a server and sent to another doctor. The input display data that includes the medical image data, would be analyzed to recreate the augmented positions of the augmented data. Such as, displaying the medical image determined to be the main image that shows the diagnosis with the annotations of various doctors. Thus, Kim teaches determine, based on analysing the one or more images comprising one or more pixels of the input display data, at least one augmentation position for displaying the augmentation data at the least one display, wherein at least one of the augmentation data and the at least one augmentation position is determined based on analysing an informational content of the user interface data.
Regarding claim 2, Ohyu teaches the processing device (Medical Processing Apparatus 1, Para. 0055) according to claim 1, wherein the processing device is couplable (Fig. 1) between the image rendering device (CT Apparatus or MRI Apparatus, Para. 0060) and the at least one display. (Display 14, Para. 0056)
Regarding claim 4, Ohyu teaches the processing device according to claim l,
wherein the input display data is rendered by the image rendering device;
and/or wherein the input display data (Image Data, Para. 0083) is displayable at the at least one display. (Display 14, Para. 0061 and 0068, Fig. 1)
Regarding claim 5, Ohyu fails to explicitly teach the processing device according to claim l, wherein the input display data and/or the output display data includes one or more image frames displayable at the at least one display.
However, Kim teaches the processing device according to claim l, wherein the input display data (Image File) and/or the output display data (Medical Information) includes one or more image frames displayable at the at least one display. (Page 4, Para. 4) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Input/Output Display Data to incorporate Kim’s Image Frames. Since doing so would provide the benefit of displaying information to a display. Displays work by displaying frames to a display (Screen, Monitor, and etc.). Hence, one of ordinary skill in the art would recognize the need for image frames to have a functional display.
Regarding claim 6, Ohyu teaches the processing device according to claim l,
wherein the input display data (Image Data) includes textual and/or numerical information; (Para. 0082-0083)
However, Ohyu fails to explicitly teach:
and wherein the one or more processors is configured to determine at least one of the augmentation data and the at least one augmentation position based on extracting the textual and/or numerical information from the input display data.
Kim teaches:
and wherein the one or more processors (Medical Data Processing Apparatus, Page 5, Para. 5) is configured to determine at least one of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) and the at least one augmentation position (Position of Image Frame, Page 4, Para. 4) based on extracting the textual and/or numerical information from the input display data. (Medical Information, Page. 21, Para. 2-5)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Augmentation Data and Augmentation Position. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary and configuring the augmented data in an easily readable form is ideal for medical devices.
Regarding claim 8, Ohyu teaches the processing device according to claim 1, further comprising:
a communication circuitry communicatively couplable to an external data source; (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data ((Display only Necessary Patterns, Para. 0083) from the external data source. (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
Regarding claim 9, Ohyu teaches the processing device according to claim l, wherein the augmentation data (Image Data for Lesions, Patterns in Data), includes medical data, video data, medical video data, medical image data, image data, or a combination thereof. (Para. 0052 and 0083)
Regarding claim 10, Ohyu teaches the processing device according to claim l,
wherein the user interface data includes one or more of at least one information item (Patient Data and Target Data), at least one state information item, and at least one control item of the graphical user interface; (Fig. 10)
However, Ohyu fails to explicitly teach:
and wherein the one or more processors is configured to determine at least one of the augmentation data and the at least one augmentation position based on extracting, from the user interface data, one or more of the following graphical user interface items: the at least one information item, the at least one state information item, and the at least one control item.
Kim teaches:
and wherein the one or more processors is configured to determine at least one of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) and the at least one augmentation position (Position of Image Frame, Page 4, Para. 4) based on extracting, from the user interface data, one or more of the following graphical user interface items: the at least one information item, (Medical Image Receiving Unit 120) the at least one state information item, and the at least one control item (Group Tabs). (Fig. 17, Page 10 Para. 8 and Page 11 Para. 1)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Augmentation Data and Augmentation Position based on User Interface Data. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary, configuring the augmented data in an easily readable form is ideal for medical devices. As well as being able to interact with the data allows medical professionals to make diagnosis and view all data related to a patient.
Regarding claim 11, Ohyu teaches the processing device according to claim 10, further comprising:
a communication circuitry communicatively couplable to an external data source; (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
However, Ohyu fails to explicitly teach:
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data from the external data source based on one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, the at least one state information item, and the at least one control item.
Kim teaches:
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) from the external data source (Patient Information Server, Page 3, Para. 5) based on one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, (Medical Image Receiving Unit 120) the at least one state information item, and the at least one control item (Query Bar 170). (Fig. 17, Page 10 Para. 8 and Page 11 Para. 1)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Augmentation Data based on User Interface Data items. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary, configuring the augmented data in an easily readable form is ideal for medical devices. As well as being able to interact with the data allows medical professionals to make diagnosis and view all data related to a patient.
Regarding claim 12, Ohyu fails to explicitly teach the processing device according to claim 11, wherein the one or more processors is configured to retrieve the at least part of the augmentation data from the external data source based on comparing one or more data items stored at the external data source with one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, the at least one state information item, and the at least one control item.
However, Kim teaches the processing device according to claim 11, wherein the one or more processors is configured to retrieve the at least part of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) from the external data source (Patient Information Server, Page 3, Para. 5) based on comparing one or more data items (Medical Record, Page 6, Para. 1) stored at the external data source (Patient Information Server, Page 3, Para. 5) with one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, (Medical Image Receiving Unit 120) the at least one state information item, and the at least one control item (Query Bar 170). (Fig. 17, Page 10 Para. 8 and Page 11 Para. 1)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Augmentation Data based on Comparing User Interface Data items and Data Items Stored at an External Source. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data by comparing data items and only displaying one’s that match. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary, configuring the augmented data in an easily readable form is ideal for medical devices. As well as being able to interact with the data allows medical professionals to make diagnosis and view all data related to a patient.
Regarding claim 13, Ohyu teaches the processing device according to claim l, wherein the augmentation data (Display only Necessary Patterns, Para. 0083) and/or the output display data (Patterns extracted by Extracting Function 22, Para. 0068) includes query data indicative of a query prompting a user to confirm correctness of one or more of the at least one information item (Image-Interpreter can Confirm Negative or Positive if Classification is Correct, Para. 0082), the at least one state information item, and the at least one control item of the graphical user interface extracted from the user interface data (Displaying data, Para. 0059) of the input display data. (Image Data, Para. 0083) (Once calculations have been performed to determine patterns in image, the image-interpreted is prompted to confirm if the classification of the pattern is correct or not. Para. 0076 and 0082)
Regarding claim 15, Ohyu teaches the processing device according to claim 10, wherein the one or more processors (Processing Circuitry 11, Para. 0057) is configured to determine a change of one or more of the at least one information item (Measured Value Graph, Fig. 20A – Fig. 20D), the at least one state information item and the at least one control item of the graphical user interface based on comparing the input display data (Fig. 20B) with previous input display data (Fig. 20A) preceding the input display data in time. (The operator of the apparatus can change the threshold values of the graphs present in the display. The graphs are then updated as the threshold values change. Para. 0309-0311 Fig. 20A – Fig. 20D)
Regarding claim 16, Ohyu teaches the processing device according to claim l, wherein the graphical user interface (GUI, Para. 0059) indicated by the user interface data relates to a patient management system (Fig.10) and/or contains information about one or more patients (Fig.10, Fig. 20A – Fig.20D), about a medical condition of one or more patients (Lesions, Para. 0066-0067), and/or about a medical treatment of one or more patients.
Regarding claim 20, Ohyu fails to explicitly teach the processing device according to claim l,
wherein the augmentation position is a position within the graphical user interface indicated by interface data contained in the input display data;
and/or wherein the one or more processors is configured to detect the graphical user interface based on analysing the input display data.
However, Kim teaches the processing device according to claim l,
wherein the augmentation position is a position within the graphical user interface indicated by interface data contained in the input display data;
and/or wherein the one or more processors is configured to detect the graphical user interface based on analysing the input display data. (Reproducing Medical Information or Image File on a Display using a Graphical User Interface, Page 4, Para. 4)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Graphical User Interface. Since doing so would provide the benefit of displaying medical data to a graphical user interface, with the graphical user interface tailored to displaying medical data. Graphical user interfaces are designed to display the type of data the graphical user interface will be displaying and have standard layouts to them.
Regarding claim 21, Ohyu fails to explicitly teach the processing device according to claim l,
wherein the augmentation position is a position within a predefined window or region indicated by the input display data;
and/or wherein the one or more processors is configured to detect a predefined window or region based on analysing the input display data.
However, Kim teaches the processing device according to claim l,
wherein the augmentation position is a position within a predefined window (Display Windows One Through Four) or region indicated by the input display data; (Page 4, Para. 4)(Each window is assigned specific data to display. Window one and two display medical information and window three and four display server lists.)
and/or wherein the one or more processors is configured to detect a predefined window or region based on analysing the input display data.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s processors to incorporate Kim’s Graphical User Interface Windows. Since doing so would provide the benefit of displaying medical data to a graphical user interface, with the graphical user interface tailored to displaying medical data. Graphical user interfaces are designed to have layouts where each section of the interface is assigned to display a specific part of the data. There are common layout designs for graphical user interfaces based on what is to be displayed.
Regarding claim 23, Ohyu fails to explicitly teach the processing device according to claim l,
wherein the one or more processors is configured to detect a command input from a user, the command input being visually displayable at the at least one display;
and wherein the one or more processors is configured to determine at least one of the augmentation data and the augmentation position based on the detected command input.
However, Kim teaches the processing device according to claim l,
wherein the one or more processors is configured to detect a command input (Query Box 170 or Scoring Box 180) from a user, the command input being visually displayable at the at least one display; (Fig.16 and Fig. 17)
and wherein the one or more processors is configured to determine at least one of the augmentation data and the augmentation position based on the detected command input. (Group Tab #1 or #2, Fig. 16 and Fig. 17)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Display to incorporate Kim’s Command Input. Since doing so would provide the benefit of letting a user interact with the displayed data. Graphical user interfaces are made to be interacted with by the user.
Regarding claim 24, Ohyu fails to explicitly teach the processing device according to claim 23, wherein the one or more processors is configured to detect the command input based on identifying one or more of a predefined text input from the user (Query Box 170 or Scoring Box 180, Fig. 16 and Fig. 17), a predefined numerical input from the user (Query Box 170 or Scoring Box 180, Fig. 16 and Fig. 17), a predefined cursor movement performed by the user (Using the Cursor to use the Scroll Bars, Fig. 16 and Fig. 17), a predefined click operation performed by the user (Clicking on Group #1 Tab, Fig. 16 and Fig. 17), a predefined control operation performed by the user at the image rendering device (Annotations Made By Other Doctors, Page 6, Para. 4), and a predefined object displayed at the at least one display. (Images Displayed in Windows 110 or 130, Fig. 16 and Fig. 17) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s User Inputs Since doing so would provide the benefit of letting a user interact with the displayed data. Graphical user interfaces are made to be interacted with by the user.
Regarding claim 25, Ohyu fails to explicitly teach the processing device according to claim 23, wherein the command input is a persistent command input or a transient command input provided by the user based on controlling the image rendering device.
However, Kim teaches the processing device according to claim 23, wherein the command input (Annotation Generation Unit, Doctors Making Annotations) is a persistent command input or a transient command input provided by the user based on controlling the image rendering device. (Server or Other Client Device, Page 6, Para. 3-4) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Persistent Command Input. Since doing so would provide the benefit of saving persisting changes made by the user through a command input. When a user interacts with a graphical user interface, changes made to the data displayed can be temporary or permanent. Making some changes persistent is desirable when modifying data.
Regarding claim 26, Ohyu fails to explicitly teach the processing device according to claim 23, wherein the one or more processors is configured to determine at least one control function associated with the detected command input.
However, Kim teaches the processing device according to claim 23, wherein the one or more processors is configured to determine at least one control function (Generate Medical Image Data and Upload to Server, Page 19 Para. 4-8) associated with the detected command input. (Input Medical Information, Fig. 15) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Command Input and Control Functions. Since doing so would provide the benefit of letting a user interact with the displayed data. Graphical user interfaces are made to be interacted with by the user and contain various modalities that let the user interact with them. These interactions can modify the data resulting in a control function being determined, such as a saving menu, uploading menu, exporting menu, and etc....
Regarding claim 27, Ohyu fails to explicitly teach the processing device according to claim 26, wherein one or more processors is further configured to perform the determined at least one control function based on controlling the processing device associated and/or based on controlling one or more external devices communicatively and/or operatively coupled to the processing device.
However, Kim teaches the processing device according to claim 26, wherein one or more processors is further configured to perform the determined at least one control function (Generate Medical Image Data and Upload to Server, Page 19 Para. 4-8) based on controlling the processing device (Medical Image Data Processing, Page 4, Para. 3) associated and/or based on controlling one or more external devices communicatively and/or operatively coupled to the processing device. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Control Functions that Control the Medical Data Processing Device. Since doing so would provide the benefit of controlling the medical data processing device through control functions initiated from the graphical user interface. Graphical user interfaces are made to be interacted with by the user and contain various modalities that let the user interact with them. These interactions can modify the medical data resulting in a control function being determined, such as a saving menu, uploading menu, exporting menu, and etc...
Regarding claim 28, Ohyu fails to explicitly teach the processing device according to claim 26, wherein the at least one control function includes one or more of recording the input display data, taking a screenshot of the display data, analyse content of the display data, display one or more control elements at the at least one display, retrieving data from one or more external sources, and controlling one or more medical devices couplable to the processing device.
However, Kim teaches the processing device according to claim 26, wherein the at least one control function includes one or more of recording the input display data, taking a screenshot of the display data, analyse content of the display data, display one or more control elements at the at least one display (Selecting a Server from Server List 140, Fig. 16 and Fig. 17), retrieving data from one or more external sources (Receive Information from Patient Information Server, Page 3, Para. 5), and controlling one or more medical devices couplable to the processing device. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus to incorporate Kim’s Control Functions. Since doing so would provide the benefit of controlling the medical data processing device through control functions initiated from the graphical user interface. Graphical user interfaces are made to be interacted with by the user and contain various modalities that let the user interact with them. These interactions can modify the medical data resulting in a control function being determined, such as a saving menu, uploading menu, exporting menu, and etc...
Regarding claim 30, Ohyu teaches a system (Para. 0313) for analysing and/or augmenting display data (Para. 0055 and Fig. 1), comprising:
a processing device (Medical Data Processing Apparatus 1, Para. 0056) having at least one processor executing instructions causing the at least one processor to: (Fig. 1)
determine, based on analysing (Examination of Image Data Belongs to Pattern, Para. 0083) input display data that includes one or more images comprising one or more pixels, (Image data, Para. 0083), augmentation data for augmenting (Display only Necessary Patterns, Para. 0083) the input display data; (Image data, Para. 0083)
an image rendering device (CT Apparatus or MRI Apparatus, Para. 0060) for providing input display data (Image Data, Para. 0083) to the processing device;
at least one display (Display 14, Para. 0059) for displaying output display data (Patterns extracted by Extracting Function 22, Para. 0068) provided by the processing device; (Output Function 23, Para. 0068)
one or more computing devices (Network Interface 15, Para. 0063) for providing data or information displayable (Displaying data, Para. 0059) at the at least one display; (Display 14, Para. 0059)
and one or more external data sources (Network Interface 15, Para. 0063) for providing at least a part of the augmentation data. (Image Data, Para. 0083)
However, Ohyu fails to teach:
determine, based on analysing the one or more images comprising one or more pixels of the input display data, at least one augmentation position for displaying the augmentation data at the least one display, wherein at least one of the augmentation data and the at least one augmentation position is determined based on analysing an informational content of the user interface data;
and generate the output display data based on supplementing the input display data with the determined augmentation data and the determined at least one augmentation position, such that the output display data is displayable at the least one display;
Kim teaches
determine, based on analysing the one or more images (Medical Images, Page 3, Para. 1-3) comprising one or more pixels of the input display data (Medical Information and Image Frame, Page 4, Para. 4), at least one augmentation position (Position of Image Frame, Page 4, Para. 4) for displaying the augmentation data at the least one display, wherein at least one of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) and the at least one augmentation position (Position of Image Frame, Page 4, Para. 4) is determined based on analysing an informational content (Based on Display Window Content, Page 4, Para. 4) of the user interface data (Graphical User Interface, Page 4, Para. 4); and (Fig. 15-17)
generate the output display data based on supplementing the input display data (Medical Data, Page 17, Para. 6) with the determined augmentation data (Outputting Selected Medical Images Based on Buttons, Page 17, Para. 6) and the determined at least one augmentation position (Position of Image Frame, Page 4, Para. 4), such that the output display data is displayable at the least one display. (Fig.17) (Depending on the display window different text corresponding to medical information and image frames are displayed at different positions or grouped together. Page 4, Para. 4 The user can then interact with the user interface in many ways to view the medical data through buttons and tabs. Page 17, Para. 6, Fig. 17)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus’s to incorporate Kim’s User Interface. Since, doing so would provide the benefit of following a common standard. The ability to manipulate data using a graphical user interface or determine where in the graphical user interface data is placed based on the data is a common standard in graphical user interfaces. Since, graphical user interfaces are built to be user-friendly and interactive. Augmenting the data and position is a frequent occurrence in user interfaces by various modalities (Clicking, Moving Windows, Search Bars, Inputting Data, and etc.)
Additionally, as discussed above, Kim teaches a medical image data processing system. The system receives encoded medical data which includes medical image data. The medical data is then extracted/decoded and reproduced onto a screen (Page 4, Section: Tech-Solution Para. 1-3). The medical data received can be unmodified data directly from a device that captures the medical data and modified data by doctors like annotations or diagnosis (Page. 6, Para. 2-4). Medical Images can be displayed individually or together (Fig. 16 and Fig. 17). The medical images received by the system can be classified into groups (Page 9, Para. 6) and can specify a main image for a specific diagnosis (Page 3, Para. 4-6). A doctor can modify and move the medical images within the display to specific positions for diagnosis (Page 21, Para. 2-5). Kim teaches the medical data can be shared remotely to many doctors to reconfirm the diagnosis or to get other doctor’s opinions (Page 6, Para. 4). Because of this when the medical data is extracted/decoded from a server and sent to another doctor. The input display data that includes the medical image data, would be analyzed to recreate the augmented positions of the augmented data. Such as, displaying the medical image determined to be the main image that shows the diagnosis with the annotations of various doctors. Thus, Kim teaches determine, based on analysing the one or more images comprising one or more pixels of the input display data, at least one augmentation position for displaying the augmentation data at the least one display, wherein at least one of the augmentation data and the at least one augmentation position is determined based on analysing an informational content of the user interface data.
Regarding method claim 31, is drawn to the method of using the corresponding device claimed in claim 1. Therefore, method claim 31 corresponding to device claim 1 is rejected for the same reasons of obviousness as used above.
Regarding claim 33, Kim teaches a non-transitory computer-readable medium storing instructions (Para. 0051) Therefore, claim 33 instructions which, when executed by at least one processor is substantially equivalent to the processor configuration in process device of claim 1 and is rejected for the same reasons of obviousness as used in claim 1.
Claim(s) 3, 7, 14, 17-19, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Ohyu et al. U.S. Patent Application Publication 20210407655 A1 in view of Kim et al. WIPO International Application 2010120061 A2 in further view of Cohen et al. U.S. Patent Application Publication 20200250826 A1.
Regarding claim 3, Ohyu and Kim fail to explicitly teach the processing device according to claim 1, wherein the one or more processors is configured to analyse the input display data and generate the output display data in real time;
and/or wherein the one or more processors is configured to analyse the input display data and generate the output display data with a latency non-perceptible by a user.
Ohyu, Kim, and Cohen are analogous to the claimed invention because all of them are in the same field of developing medical data processing apparatuses that are used to process medical data.
Cohen teaches the processing device (Medical Data Processing Device, Para. 0059) according to claim 1,
wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to analyse the input display data (Medical Data, Para. 0059) and generate the output display data (Graphics and Images, Para. 0047) in real time;
and/or wherein the one or more processors is configured to analyse the input display data and generate the output display data with a latency non-perceptible by a user.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Output Display Data altered by Kim to incorporate Cohen’s real-time processing. Since doing so would provide the benefit of creating a medical data processing apparatus, that processes data in real-time. Technology that requires user interaction often processes data in real-time to allow the user a seamless experience, when interacting with the technology to avoid wait times.
Regarding claim 7, Ohyu and Kim fail to explicitly teach the processing device according to claim 6, wherein the one or more processors is configured to extract the textual and/or numerical information from the input display data based on optical character recognition.
However, Cohen teaches the processing device (Medical Data Processing Device, Para. 0059) is according to claim 6, wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to extract the textual and/or numerical information from the input display data (Medical Data, Para. 0059) based on optical character recognition. (Para. 0043 and 0095) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s processor altered by Kim to incorporate Cohen’s Optical Character Recognition. Since doing so would provide the benefit of configuring the processor to be able to process data that otherwise it wouldn’t have been able to. Medical data comprises a wide variety of forms such as text, image, and video. Hence, optical character recognition can be used to retrieve data in other forms that would otherwise have to be manually inputted.
Regarding claim 14, Ohyu and Kim fail to explicitly teach the processing device according to claim 13, wherein the one or more processors is further configured to determine a response to the user’s query based on analysing further input display data received subsequent to the input display data.
However, Cohen teaches the processing device (Medical Data Processing Device, Para. 0059) according to claim 13, wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is further configured to determine a response to the user’s query based on analysing further input display data received subsequent to the input display data. (A graphical overlay can be displayed to alert the user of an incorrect error in the data and the recommended correction, Para. 0141. To recommend a correction subsequent input display data would have to be retrieved to cross-compare what incorrect information was inserted to respond with a recommendation.) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor altered by Kim to incorporate Cohen’s Response to a Query. Since doing so would provide the benefit of correcting errors found in the information in a quick and easy manner. Errors present in medical data can lead to serious consequences, such as confusing patient data or misdiagnosing. Hence, being able to quickly and efficiently correct errors that are prompted to the user is important.
Regarding claim 17, Ohyu and Kim fail to explicitly teach the processing device according to claim l,
wherein the user interface data includes a patient identification item for uniquely identifying a patient;
and wherein the one or more processors is configured to extract the patient identification item from the user interface data to determine at least one of the augmentation data or the at least one augmentation position.
However, Cohen teaches the processing device according to claim l,
wherein the user interface data includes a patient identification item for uniquely identifying a patient; (Patient ID, Para. 0122 and 0033)
and wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to extract the patient identification item (Patient ID) from the user interface data to determine at least one of the augmentation data (Patient Medical Records) or the at least one augmentation position. (Location in Medical UI of Medical Monitor and EMR Application, Para. 0033 and 0122)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus altered by Kim to incorporate Cohen’s Patient Identification Item. Since doing so would provide the benefit of uniquely tracking each patient’s medical record, as well as grouping all related patient information by tying it to a unique ID. This allows the apparatus to easily retrieve all data associated to a patient by using the unique ID. The method of using a unique ID to identify information about a user/patient is standard practice.
Regarding claim 18, Ohyu teaches the processing device according to claim 17, further comprising:
a communication circuitry communicatively couplable to an external data source; (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
However, Ohyu and Kim fail to explicitly teach:
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data from the external data source based on the extracted patient identification item.
Cohen teaches:
and wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to retrieve at least a part of the augmentation data (Medical data) from the external data source (EMR Application) based on the extracted patient identification item. (Patient ID are compared to identify medical data correspond to the monitor UI, Para. 0033 and 0122)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus altered by Kim to incorporate Cohen’s Patient Identification Item. Since doing so would provide the benefit of uniquely tracking each patient’s medical record, as well as grouping all related patient information by tying it to a unique ID. This allows the apparatus to easily retrieve all data associated to a patient by using the unique ID. The method of using a unique ID to identify information about a user/patient is standard practice.
Regarding claim 19, Ohyu teaches the processing device (Medical Data Processing Apparatus, Para. 0055-0056) according to claim 18, wherein the augmentation data retrieved from the external data source ((Network Interface 15, Para. 0055-0056 and 0063, Fig. 1) includes medical data (Medical Images, Para. 0060) associated with the patient.
Regarding claim 22, Ohyu and Kim fail to explicitly teach the processing device according to claim l, wherein the one or more processors is configured to detect , based on a color, the predefined window or region.
However, Cohen teaches the processing device according to claim l, wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to detect, based on a color (Distinct Color or Inverted Color, or Highlighting), the predefined window or region (Graphical Overlay). (Para. 0141) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Graphical User Interface altered by Kim’s Windows to incorporate Cohen’s Coloring and Detection of Windows. Since doing so would provide the benefit of detecting windows based on colors as well as positions. In the case of Cohen colors, symbols, and comments are used to alert the user of an important overlay. These changes are naturally tracked by the graphical user interface. Detecting these overlays, signals to the graphical user interface if an important alert has been issued to the user. As well as if this application is reloaded at a later date, the graphical user interface understands to redisplay the overlay with the specific color, symbol, or comment.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIANNA R COCHRAN whose telephone number is (571)272-4671. The examiner can normally be reached Mon-Fri. 7:30am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIANNA RENAE COCHRAN/Examiner, Art Unit 2615 /ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights