DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Arguments
This is in response to applicant’s amendment/response filed on 01/28/2026 which have been entered and made of record.
Applicant’s arguments regarding claim rejections under 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant argues the Applicant's attorney submits that the relied upon aspects of Kim are wholly silent with respect to "determin[ing] ... a subset of pixels corresponding to at least one augmentation position for displaying the augmentation data", where the "subset of pixels corresponding to at least one augmentation position" are determined "from among the plurality pixels of the image of the input display data", much less where the determined "subset of pixels corresponding to at least one augmentation position" are determined "relative to at least one item" captured in "the image of the input display data" as set forth in amended independent claims.
Rather, from inspection of the relied upon description and FIGS of Kim, the Applicant's attorney submits that Kim's "display window[s]" are fixed. Accordingly, while Kim's "medical image[s]", "text of corresponding medial information", and "server list[s]" utilized to populate the "display window[s]" may vary, the Applicant's attorney submits that the "position" of the "display window[s]" themselves are fixed and are not determined as the "subset of pixels corresponding to at least one augmentation position" as particularly set forth in the independent claims.
As another example, in rejecting independent claim 1 (pre-amendment), the Office Action alleges that Page 17, Para. 6, Page 4, Para. 4, and FIG. 17 of Kim disclose "generate the output display data based on supplementing the input display data with the determined augmentation data and the determined at least one augmentation position, such that the graphical user inter face is displayable at the least one display and includes the image capturing the at least one item augmented with the determined augmentation data at the subset of pixels corresponding to the determined at least one augmentation position"
Initially, the Applicant's attorney submits that the relied upon aspects of Kim fail to disclose at least the above features based on Kim's failure to disclose at least "the determined at least one augmentation position" for the reasons set forth above.
Nonetheless, as noted above, relied upon Page 4, Para. 4 of Kim discloses an "image data viewer" that "reproduces medical information and the image file ... through a graphical user interface ... configured to display at least one image frame according to the reproduction of the image file." Further. relied upon Page 17, Para. 6 of Kim discloses "output ... for medical images" that is based on a "final diagnosis" used to modify "medical data" and various "report[s]" can be provided through interacting with different "button[s]".
Put another way, the relied upon aspects of Kim disclose that a doctor can add a "final diagnosis", the "medical data" can be updated, and various "report[s]" can be provided by the doctor interacting with different "button[s]".
However, the Applicant's attorney submits that the relied upon aspects of Kim are irrelevant to "generat[ing] the output display data ..., such that the graphical user interface is displayable at the least one display and includes the image capturing the at least one item augmented with the determined augmentation data at the subset of pixels corresponding to the determined at least one augmentation position". Rather, the relied upon aspects of Kim are all based on different user interactions by the doctor and is not "an automated detection of what augmentation data or information is to be displayed at which position of the display" as described in paragraph [0021] of the Present Application, and as described in the amended independent claims.
Examiner respectable disagrees, While the display windows and image frames can be fixed in placed or moveable. Kim teaches the content inside the display windows or image frames is changed based on the data currently being displayed (Page 4 Para. 3 and Page 21 Para. 2-7) which modifies the associated pixel data. For example Fig. 16 and Fig. 17 showcase a first display window 1100 containing different content (Several Medical Images in a Grid or a Single Medical Image, Page 21 Para. 2-3). These medical images utilize different subsets of pixels associated with the first display window 1100. A user selecting a single medical image to be enlarged (Page 21 Para. 3, Fig. 17) modifies the position of the medical image and the pixels associated with the medical image position. As well as when an image is enlarged new pixels can be created, stretched, or scaled based on how the image is enlarged which can modify pixel data. Thus, Kim teaches subset of pixels(Pixels Associated with New Position of the Medical Images) corresponding to at least one augmentation position (Position of Image in the Image Frame).
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “an automated detection of what augmentation data or information is to be displayed at which position of the display") are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Regarding the remaining arguments applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth below.
Claim Objections
Claim 4 is objected to because of the following informalities: Claim 4 does not indicate which claim it depends from. Examiner is interpreting claim 4 to depend from claim 1. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-6, 8-13, 15-16, 20-28, 30-31, and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Ohyu et al. U.S. Patent Application Publication 20210407655 A1 in view of Kim et al. WIPO International Application 2010120061 A2.
Regarding claim 1, Ohyu teaches a processing device (Medical Data Processing Apparatus 1, Para. 0056) for analysing and/or augmenting display data (Para. 0055), comprising (Fig. 1):
an input interface (Input Interface 13, Para. 0061) configured to receive input display data (Display Data including Image Data, Para. 0083) from an image rendering device (CT Apparatus or MRI Apparatus, Para. 0060);
an output interface (Output Function 23, Para. 0068) configured to transmit output display data (Patterns extracted by Extracting Function 22, Para. 0068) to at least one display (Display 14, Para. 0068), wherein the input display data (Display Data including Image Data, Para. 0083) includes user interface data (Displaying data, Para. 0059) indicative of an image (Displaying Data including Image data, Reference Data, or any Data to be Displayed) corresponding to a graphical user interface (GUI, Para. 0059) displayable at the at least one display (Display 14, Para. 0059)¸wherein the image captures at least one item(Displaying Data including Image data, Reference Data, or any Data to be Displayed) of the graphical user interface (GUI, Para. 0059), and wherein the image(Displaying Data including Image data, Reference Data, or any Data to be Displayed)includes a plurality of pixels; The input display data contains the data to be displayed such as an image. This input display data contains a plurality of pixels associated with the data to be displayed. As the input display data is received from an image rendering device, which to render an image pixels are required.
and one or more processors (Processing Circuitry 11, Para. 0057) configured to:
determine, based on analysing (Examination of Image Data Belongs to Pattern, Para. 0083) the user interface data(Displaying data, Para. 0059) of the input display data (Display Data including Image Data, Para. 0083), augmentation data for augmenting (Display only Necessary Patterns, Para. 0083) the input display data; (Display Data including Image Data, Para. 0083) The data to be displayed is analyzed to only display image data necessary for a specific pattern, Para. 0083.
However, Ohyu fails to explicitly teach:
determine, from among the plurality of pixels of the image of the input display data, a subset of pixels corresponding to at least one augmentation position for displaying the augmentation data at the graphical user interface and relative to the at least one item; and
generate the output display data based on supplementing the input display data with the determined augmentation data and the determined at least one augmentation position, such that the graphical user interface is displayable at the least one display and includes the image capturing the at least one item augmented with the determined augmentation data at the subset of pixels corresponding to the determined at least one augmentation position.
Ohyu and Kim are analogous to the claimed invention because both of them are in the same field of developing medical data processing apparatuses that are used to process medical image data.
Kim teaches:
determine, from among the plurality of pixels (Pixels Associated with the Current Position of the Medical Images) of the images (Medical Images, Page 3, Para. 1-3) of the input display data (Medical Information and Image Frame, Page 4, Para. 4), a subset of pixels (Pixels Associated with New Position of the Medical Images) corresponding to at least one augmentation position (Position of Image in the Image Frame, Page 4, Para. 4) for displaying the augmentation data at the graphical user interface(Graphical User Interface, Page 4, Para. 4) and relative to the at least one item (Any Graphical User Interface Objects – Group Tabs, Buttons, Image Frames, Images, Text, etc…); and (Fig. 15-17) As stated above Kim teaches the content inside the display windows or image frames is changed based on the data currently being displayed (Page 4 Para. 3 and Page 21 Para. 2-7) which modifies the associated pixel data. For example Fig. 16 and Fig. 17 showcase a first display window 1100 containing different content (Several Medical Images in a Grid or a Single Medical Image, Page 21 Para. 2-3). These medical images utilize different subsets of pixels associated with the first display window 1100. A user selecting a single medical image to be enlarged (Page 21 Para. 3, Fig. 17) modifies the position of the medical image and the pixels associated with the medical image position. As well as when an image is enlarged new pixels can be created, stretched, or scaled based on how the image is enlarged which can modify pixel data. Thus, Kim teaches subset of pixels(Pixels Associated with New Position of the Medical Images) corresponding to at least one augmentation position (Position of Image in the Image Frame).
generate the output display data based on supplementing the input display data (Medical Data, Page 17, Para. 6) with the determined augmentation data (Outputting Selected Medical Images Based on Buttons, Page 17, Para. 6) and the determined at least one augmentation position (Position of Image in the Image Frame, Page 4, Para. 4), such that the graphical user interface(Graphical User Interface, Page 4, Para. 4) is displayable at the least one display,(Fig.17) (Depending on the display window different text corresponding to medical information and image frames are displayed at different positions or grouped together. Page 4, Para. 4 The user can then interact with the user interface in many ways to view the medical data through buttons and tabs. Page 17, Para. 6, Fig. 17) and includes the image(Medical Data/Images, Page 17, Para. 6) capturing the at least one item(Any Graphical User Interface Objects – Group Tabs, Buttons, Image Frames, Images, Text, etc…) augmented with the determined augmentation data(Outputting Selected Medical Images Based on Buttons, Page 17, Para. 6) at the subset of pixels(Pixels Associated with New Position of the Medical Images) corresponding to the determined at least one augmentation position(Position of Image in the Image Frame, Page 4, Para. 4).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus’s Graphical User Interface to incorporate Kim’s Graphical User Interface. Since, doing so would provide the benefit of following a common standard. The ability to manipulate data using a graphical user interface or determine where in the graphical user interface data is placed based on the data is a common standard in graphical user interfaces. Since, graphical user interfaces are built to be user-friendly and interactive. Augmenting the data and position is a frequent occurrence in user interfaces by various modalities (Clicking, Moving Windows, Search Bars, Inputting Data, and etc.)
Regarding claim 2, Ohyu teaches the processing device (Medical Processing Apparatus 1, Para. 0055) according to claim 1, wherein the processing device is couplable (Fig. 1) between the image rendering device (CT Apparatus or MRI Apparatus, Para. 0060) and the at least one display. (Display 14, Para. 0056)
Regarding claim 4, Ohyu teaches the processing device according to claim l,
wherein the input display data is rendered by the image rendering device;
and/or wherein the input display data (Image Data, Para. 0083) is displayable at the at least one display. (Display 14, Para. 0061 and 0068, Fig. 1)
Regarding claim 5, Ohyu fails to explicitly teach the processing device according to claim l, wherein the input display data and/or the output display data includes one or more image frames, including at least the image, displayable at the at least one display.
However, Kim teaches the processing device according to claim l, wherein the input display data (Image File) and/or the output display data (Medical Information) includes one or more image frames, including at least the image(Medical Data/Images, Page 17, Para. 6), displayable at the at least one display. (Page 4, Para. 4) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Input/Output Display Data to incorporate Kim’s Image Frames. Since doing so would provide the benefit of displaying information to a display. Displays work by displaying frames to a display (Screen, Monitor, and etc.). Hence, one of ordinary skill in the art would recognize the need for image frames to have a functional display.
Regarding claim 6, Ohyu teaches the processing device according to claim l,
wherein the at least one item (Displaying Data including Image data, Reference Data, or any Data to be Displayed) includes textual and/or numerical information; (Para. 0082-0083)
However, Ohyu fails to explicitly teach:
and wherein the one or more processors is configured to determine at least one of the augmentation data and the at least one augmentation position based on extracting the textual and/or numerical information from the at least one item.
Kim teaches:
and wherein the one or more processors (Medical Data Processing Apparatus, Page 5, Para. 5) is configured to determine at least one of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) and the at least one augmentation position (Position of Image Frame, Page 4, Para. 4) based on extracting the textual and/or numerical information from the at least one item(Any Graphical User Interface Objects – Group Tabs, Buttons, Image Frames, Images, Text, etc…). (Medical Information, Page. 21, Para. 2-5)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Augmentation Data and Augmentation Position. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary and configuring the augmented data in an easily readable form is ideal for medical devices.
Regarding claim 8, Ohyu teaches the processing device according to claim 1, further comprising:
a communication circuitry communicatively couplable to an external data source; (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data ((Display only Necessary Patterns, Para. 0083) from the external data source. (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
Regarding claim 9, Ohyu teaches the processing device according to claim l, wherein the augmentation data (Image Data for Lesions, Patterns in Data), includes medical data, video data, medical video data, medical image data, image data, or a combination thereof. (Para. 0052 and 0083)
Regarding claim 10, Ohyu teaches the processing device according to claim l,
wherein the at least one item(Displaying Data including Image data, Reference Data, or any Data to be Displayed) includes one or more of at least one information item (Patient Data and Target Data), at least one state information item, or at least one control item of the graphical user interface; (Fig. 10)
However, Ohyu fails to explicitly teach:
and wherein the one or more processors is configured to determine at least one of the augmentation data and the at least one augmentation position based on extracting, from the user interface data, one or more of the following graphical user interface items: the at least one information item, the at least one state information item, and the at least one control item.
Kim teaches:
and wherein the one or more processors is configured to determine at least one of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) and the at least one augmentation position (Position of Image in the Image Frame, Page 4, Para. 4) based on extracting, from the user interface data, one or more of the following graphical user interface items: the at least one information item, (Medical Image Receiving Unit 120) the at least one state information item, and the at least one control item (Group Tabs). (Fig. 17, Page 10 Para. 8 and Page 11 Para. 1)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Augmentation Data and Augmentation Position based on User Interface Data. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary, configuring the augmented data in an easily readable form is ideal for medical devices. As well as being able to interact with the data allows medical professionals to make diagnosis and view all data related to a patient.
Regarding claim 11, Ohyu teaches the processing device according to claim 10, further comprising:
a communication circuitry communicatively couplable to an external data source; (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
However, Ohyu fails to explicitly teach:
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data from the external data source based on one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, the at least one state information item, and the at least one control item.
Kim teaches:
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) from the external data source (Patient Information Server, Page 3, Para. 5) based on one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, (Medical Image Receiving Unit 120) the at least one state information item, and the at least one control item (Query Bar 170). (Fig. 17, Page 10 Para. 8 and Page 11 Para. 1)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Augmentation Data based on User Interface Data items. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary, configuring the augmented data in an easily readable form is ideal for medical devices. As well as being able to interact with the data allows medical professionals to make diagnosis and view all data related to a patient.
Regarding claim 12, Ohyu fails to explicitly teach the processing device according to claim 11, wherein the one or more processors is configured to retrieve the at least part of the augmentation data from the external data source based on comparing one or more data items stored at the external data source with one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, the at least one state information item, and the at least one control item.
However, Kim teaches the processing device according to claim 11, wherein the one or more processors is configured to retrieve the at least part of the augmentation data (Displaying Text Corresponding to Medical Information, Page 4, Para. 4) from the external data source (Patient Information Server, Page 3, Para. 5) based on comparing one or more data items (Medical Record, Page 6, Para. 1) stored at the external data source (Patient Information Server, Page 3, Para. 5) with one or more of the following graphical user interface items extracted from the user interface data: the at least one information item, (Medical Image Receiving Unit 120) the at least one state information item, and the at least one control item (Query Bar 170). (Fig. 17, Page 10 Para. 8 and Page 11 Para. 1)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Augmentation Data based on Comparing User Interface Data items and Data Items Stored at an External Source. Since, doing so would provide the benefit of being able to interact with the data being displayed and displaying information only pertinent to the data by comparing data items and only displaying one’s that match. Inventions directed to medical information have to abide by laws regarding protecting sensitive medical information. Hence, only displaying what is necessary, configuring the augmented data in an easily readable form is ideal for medical devices. As well as being able to interact with the data allows medical professionals to make diagnosis and view all data related to a patient.
Regarding claim 13, Ohyu teaches the processing device according to claim 10, wherein the augmentation data (Display only Necessary Patterns, Para. 0083) and/or the output display data (Patterns extracted by Extracting Function 22, Para. 0068) includes query data indicative of a query prompting a user to confirm correctness of one or more of the at least one information item (Image-Interpreter can Confirm Negative or Positive if Classification is Correct, Para. 0082), the at least one state information item, and the at least one control item of the graphical user interface extracted from the user interface data (Displaying data, Para. 0059) of the input display data. (Image Data, Para. 0083) (Once calculations have been performed to determine patterns in image, the image-interpreted is prompted to confirm if the classification of the pattern is correct or not. Para. 0076 and 0082)
Regarding claim 15, Ohyu teaches the processing device according to claim 10, wherein the one or more processors (Processing Circuitry 11, Para. 0057) is configured to determine a change of one or more of the at least one information item (Measured Value Graph, Fig. 20A – Fig. 20D), the at least one state information item and the at least one control item of the graphical user interface based on comparing the input display data (Fig. 20B) with previous input display data (Fig. 20A) preceding the input display data in time. (The operator of the apparatus can change the threshold values of the graphs present in the display. The graphs are then updated as the threshold values change. Para. 0309-0311 Fig. 20A – Fig. 20D)
Regarding claim 16, Ohyu teaches the processing device according to claim l, wherein the graphical user interface (GUI, Para. 0059) indicated by the user interface data relates to a patient management system (Fig.10) and/or contains information about one or more patients (Fig.10, Fig. 20A – Fig.20D), about a medical condition of one or more patients (Lesions, Para. 0066-0067), and/or about a medical treatment of one or more patients.
Regarding claim 20, Ohyu fails to explicitly teach the processing device according to claim l,
wherein the augmentation position is a position within the graphical user interface indicated by interface data contained in the input display data;
and/or wherein the one or more processors is configured to detect the graphical user interface based on analysing the input display data.
However, Kim teaches the processing device according to claim l,
wherein the augmentation position is a position within the graphical user interface indicated by interface data contained in the input display data;
and/or wherein the one or more processors is configured to detect the graphical user interface based on analysing the input display data. (Reproducing Medical Information or Image File on a Display using a Graphical User Interface, Page 4, Para. 4)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Graphical User Interface. Since doing so would provide the benefit of displaying medical data to a graphical user interface, with the graphical user interface tailored to displaying medical data. Graphical user interfaces are designed to display the type of data the graphical user interface will be displaying and have standard layouts to them.
Regarding claim 21, Ohyu fails to explicitly teach the processing device according to claim l, wherein the subset of pixels corresponding to the at least one augmentation position is: a range of pixels that is selected from among the plurality of pixels relative to the at least one item, an area of pixels that is selected from among the plurality of pixels relative to the at least one item, or pixel coordinates that are selected from among the plurality of pixels relative to the at least one item.
However, Kim teaches the processing device according to claim l, wherein the subset of pixels(Pixels Associated with New Position of the Medical Images) corresponding to the at least one augmentation position(Position of Image in the Image Frame, Page 4, Para. 4) is: a range of pixels that is selected from among the plurality of pixels relative to the at least one item, an area of pixels (Position of Images in Image Frame, Fig. 16 or Fig. 17) that is selected from among the plurality of pixels(Pixels Associated with the Current Position of the Medical Images) relative to the at least one item(Any Graphical User Interface Objects – Group Tabs, Buttons, Image Frames, Images, Text, etc…), or pixel coordinates that are selected from among the plurality of pixels relative to the at least one item. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus’s Graphical User Interface to incorporate Kim’s Graphical User Interface. Since, doing so would provide the benefit of following a common standard. The ability to manipulate data using a graphical user interface or determine where in the graphical user interface data is placed based on the data is a common standard in graphical user interfaces. Since, graphical user interfaces are built to be user-friendly and interactive. Augmenting the data and position is a frequent occurrence in user interfaces by various modalities (Clicking, Moving Windows, Search Bars, Inputting Data, and etc.)
Regarding claim 22, Ohyu and Kim fail to explicitly teach the processing device according to claim l, wherein the one or more processors is configured to select the subset of pixels(Pixels Associated with New Position of the Medical Images) to not obscure the at least one item(Any Graphical User Interface Objects – Group Tabs, Buttons, Image Frames, Images, Text, etc…). The position of the images in display windows 1100 for Fig. 16 and Fig. 17 do not obscure each other or any graphical user interface element.
Regarding claim 23, Ohyu fails to explicitly teach the processing device according to claim l,
wherein the one or more processors is configured to detect a command input from a user, the command input being visually displayable at the at least one display;
and wherein the one or more processors is configured to determine at least one of the augmentation data and the augmentation position based on the detected command input.
However, Kim teaches the processing device according to claim l,
wherein the one or more processors is configured to detect a command input (Query Box 170 or Scoring Box 180) from a user, the command input being visually displayable at the at least one display; (Fig.16 and Fig. 17)
and wherein the one or more processors is configured to determine at least one of the augmentation data and the augmentation position based on the detected command input. (Group Tab #1 or #2, Fig. 16 and Fig. 17)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Display to incorporate Kim’s Command Input. Since doing so would provide the benefit of letting a user interact with the displayed data. Graphical user interfaces are made to be interacted with by the user.
Regarding claim 24, Ohyu fails to explicitly teach the processing device according to claim 23, wherein the one or more processors is configured to detect the command input based on identifying one or more of a predefined text input from the user (Query Box 170 or Scoring Box 180, Fig. 16 and Fig. 17), a predefined numerical input from the user (Query Box 170 or Scoring Box 180, Fig. 16 and Fig. 17), a predefined cursor movement performed by the user (Using the Cursor to use the Scroll Bars, Fig. 16 and Fig. 17), a predefined click operation performed by the user (Clicking on Group #1 Tab, Fig. 16 and Fig. 17), a predefined control operation performed by the user at the image rendering device (Annotations Made By Other Doctors, Page 6, Para. 4), and a predefined object displayed at the at least one display. (Images Displayed in Windows 110 or 130, Fig. 16 and Fig. 17) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s User Inputs Since doing so would provide the benefit of letting a user interact with the displayed data. Graphical user interfaces are made to be interacted with by the user.
Regarding claim 25, Ohyu fails to explicitly teach the processing device according to claim 23, wherein the command input is a persistent command input or a transient command input provided by the user based on controlling the image rendering device.
However, Kim teaches the processing device according to claim 23, wherein the command input (Annotation Generation Unit, Doctors Making Annotations) is a persistent command input or a transient command input provided by the user based on controlling the image rendering device. (Server or Other Client Device, Page 6, Para. 3-4) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Persistent Command Input. Since doing so would provide the benefit of saving persisting changes made by the user through a command input. When a user interacts with a graphical user interface, changes made to the data displayed can be temporary or permanent. Making some changes persistent is desirable when modifying data.
Regarding claim 26, Ohyu fails to explicitly teach the processing device according to claim 23, wherein the one or more processors is configured to determine at least one control function associated with the detected command input.
However, Kim teaches the processing device according to claim 23, wherein the one or more processors is configured to determine at least one control function (Generate Medical Image Data and Upload to Server, Page 19 Para. 4-8) associated with the detected command input. (Input Medical Information, Fig. 15) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor to incorporate Kim’s Command Input and Control Functions. Since doing so would provide the benefit of letting a user interact with the displayed data. Graphical user interfaces are made to be interacted with by the user and contain various modalities that let the user interact with them. These interactions can modify the data resulting in a control function being determined, such as a saving menu, uploading menu, exporting menu, and etc....
Regarding claim 27, Ohyu fails to explicitly teach the processing device according to claim 26, wherein one or more processors is further configured to perform the determined at least one control function based on controlling the processing device associated and/or based on controlling one or more external devices communicatively and/or operatively coupled to the processing device.
However, Kim teaches the processing device according to claim 26, wherein one or more processors is further configured to perform the determined at least one control function (Generate Medical Image Data and Upload to Server, Page 19 Para. 4-8) based on controlling the processing device (Medical Image Data Processing, Page 4, Para. 3) associated and/or based on controlling one or more external devices communicatively and/or operatively coupled to the processing device. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processors to incorporate Kim’s Control Functions that Control the Medical Data Processing Device. Since doing so would provide the benefit of controlling the medical data processing device through control functions initiated from the graphical user interface. Graphical user interfaces are made to be interacted with by the user and contain various modalities that let the user interact with them. These interactions can modify the medical data resulting in a control function being determined, such as a saving menu, uploading menu, exporting menu, and etc...
Regarding claim 28, Ohyu fails to explicitly teach the processing device according to claim 26, wherein the at least one control function includes one or more of recording the input display data, taking a screenshot of the display data, analyse content of the display data, display one or more control elements at the at least one display, retrieving data from one or more external sources, and controlling one or more medical devices couplable to the processing device.
However, Kim teaches the processing device according to claim 26, wherein the at least one control function includes one or more of recording the input display data, taking a screenshot of the display data, analyse content of the display data, display one or more control elements at the at least one display (Selecting a Server from Server List 140, Fig. 16 and Fig. 17), retrieving data from one or more external sources (Receive Information from Patient Information Server, Page 3, Para. 5), and controlling one or more medical devices couplable to the processing device. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus to incorporate Kim’s Control Functions. Since doing so would provide the benefit of controlling the medical data processing device through control functions initiated from the graphical user interface. Graphical user interfaces are made to be interacted with by the user and contain various modalities that let the user interact with them. These interactions can modify the medical data resulting in a control function being determined, such as a saving menu, uploading menu, exporting menu, and etc...
Regarding claim 30, Ohyu teaches a system (Para. 0313) for analysing and/or augmenting display data (Para. 0055 and Fig. 1), comprising:
a processing device (Medical Data Processing Apparatus 1, Para. 0056) having at least one processor executing instructions causing the at least one processor to: (Fig. 1)
determine, based on analysing (Examination of Image Data Belongs to Pattern, Para. 0083) user interface data(Displaying data, Para. 0059) of input display data (Display Data including Image Data, Para. 0083), augmentation data for augmenting (Display only Necessary Patterns, Para. 0083) the input display data (Display Data including Image Data, Para. 0083 - The data to be displayed is analyzed to only display image data necessary for a specific pattern.) ,wherein the user interface data(Displaying data, Para. 0059) is indicative of an image(Displaying Data including Image data, Reference Data, or any Data to be Displayed) corresponding to a graphical user interface(GUI, Para. 0059), wherein the image(Displaying Data including Image data, Reference Data, or any Data to be Displayed) captures at least one item(Displaying Data including Image data, Reference Data, or any Data to be Displayed) , and wherein the image(Displaying Data including Image data, Reference Data, or any Data to be Displayed) includes a plurality of pixels; The input display data contains the data to be displayed such as an image. This input display data contains a plurality of pixels associated with the data to be displayed. As the input display data is received from an image rendering device, which to render an image pixels are required.
an image rendering device (CT Apparatus or MRI Apparatus, Para. 0060) for providing input display data (Image Data, Para. 0083) to the processing device;
at least one display (Display 14, Para. 0059) for displaying the output display data (Patterns extracted by Extracting Function 22, Para. 0068) provided by the processing device; (Output Function 23, Para. 0068)
one or more computing devices (Network Interface 15, Para. 0063) for providing data or information displayable (Displaying data, Para. 0059) at the at least one display; (Display 14, Para. 0059)
and one or more external data sources (Network Interface 15, Para. 0063) for providing at least a part of the augmentation data. (Image Data, Para. 0083)
However, Ohyu fails to teach:
determine, from among the plurality of pixels of the image of the input display data, a subset of pixels corresponding to at least one augmentation position for displaying the augmentation data at the graphical user interface and relative to the at least one item; and
generate the output display data based on supplementing the input display data with the determined augmentation data and the determined at least one augmentation position, such that the graphical user interface is displayable at the least one display and includes the image capturing the at least one item augmented with the determined augmentation data at the subset of pixels corresponding to the determined at least one augmentation position.
Kim teaches
determine, from among the plurality of pixels (Pixels Associated with the Current Position of the Medical Images) of the images (Medical Images, Page 3, Para. 1-3) of the input display data (Medical Information and Image Frame, Page 4, Para. 4), a subset of pixels (Pixels Associated with New Position of the Medical Images) corresponding to at least one augmentation position (Position of Image in the Image Frame, Page 4, Para. 4) for displaying the augmentation data at the graphical user interface(Graphical User Interface, Page 4, Para. 4) and relative to the at least one item (Any Graphical User Interface Objects – Group Tabs, Buttons, Image Frames, Images, Text, etc…); and (Fig. 15-17) As stated above Kim teaches the content inside the display windows or image frames is changed based on the data currently being displayed (Page 4 Para. 3 and Page 21 Para. 2-7) which modifies the associated pixel data. For example Fig. 16 and Fig. 17 showcase a first display window 1100 containing different content (Several Medical Images in a Grid or a Single Medical Image, Page 21 Para. 2-3). These medical images utilize different subsets of pixels associated with the first display window 1100. A user selecting a single medical image to be enlarged (Page 21 Para. 3, Fig. 17) modifies the position of the medical image and the pixels associated with the medical image position. As well as when an image is enlarged new pixels can be created, stretched, or scaled based on how the image is enlarged which can modify pixel data. Thus, Kim teaches subset of pixels(Pixels Associated with New Position of the Medical Images) corresponding to at least one augmentation position (Position of Image in the Image Frame).
generate the output display data based on supplementing the input display data (Medical Data, Page 17, Para. 6) with the determined augmentation data (Outputting Selected Medical Images Based on Buttons, Page 17, Para. 6) and the determined at least one augmentation position (Position of Image in the Image Frame, Page 4, Para. 4), such that the graphical user interface(Graphical User Interface, Page 4, Para. 4) is displayable at the least one display,(Fig.17) (Depending on the display window different text corresponding to medical information and image frames are displayed at different positions or grouped together. Page 4, Para. 4 The user can then interact with the user interface in many ways to view the medical data through buttons and tabs. Page 17, Para. 6, Fig. 17) and includes the image(Medical Data/Images, Page 17, Para. 6) capturing the at least one item(Any Graphical User Interface Objects – Group Tabs, Buttons, Image Frames, Images, Text, etc…) augmented with the determined augmentation data(Outputting Selected Medical Images Based on Buttons, Page 17, Para. 6) at the subset of pixels(Pixels Associated with New Position of the Medical Images) corresponding to the determined at least one augmentation position(Position of Image in the Image Frame, Page 4, Para. 4).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus’s to incorporate Kim’s User Interface. Since, doing so would provide the benefit of following a common standard. The ability to manipulate data using a graphical user interface or determine where in the graphical user interface data is placed based on the data is a common standard in graphical user interfaces. Since, graphical user interfaces are built to be user-friendly and interactive. Augmenting the data and position is a frequent occurrence in user interfaces by various modalities (Clicking, Moving Windows, Search Bars, Inputting Data, and etc.)
Regarding method claim 31, is drawn to the method of using the corresponding device claimed in claim 1. Therefore, method claim 31 corresponding to device claim 1 is rejected for the same reasons of obviousness as used above.
Regarding claim 33, Kim teaches a non-transitory computer-readable medium storing instructions (Para. 0051) Therefore, claim 33 instructions which, when executed by at least one processor is substantially equivalent to the processor configuration in process device of claim 1 and is rejected for the same reasons of obviousness as used in claim 1.
Claim(s) 3, 7, 14, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Ohyu et al. U.S. Patent Application Publication 20210407655 A1 in view of Kim et al. WIPO International Application 2010120061 A2 in further view of Cohen et al. U.S. Patent Application Publication 20200250826 A1.
Regarding claim 3, Ohyu and Kim fail to explicitly teach the processing device according to claim 1, wherein the one or more processors is configured to analyse the input display data and generate the output display data in real time;
and/or wherein the one or more processors is configured to analyse the input display data and generate the output display data with a latency non-perceptible by a user.
Ohyu, Kim, and Cohen are analogous to the claimed invention because all of them are in the same field of developing medical data processing apparatuses that are used to process medical data.
Cohen teaches the processing device (Medical Data Processing Device, Para. 0059) according to claim 1,
wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to analyse the input display data (Medical Data, Para. 0059) and generate the output display data (Graphics and Images, Para. 0047) in real time;
and/or wherein the one or more processors is configured to analyse the input display data and generate the output display data with a latency non-perceptible by a user.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Output Display Data altered by Kim to incorporate Cohen’s real-time processing. Since doing so would provide the benefit of creating a medical data processing apparatus, that processes data in real-time. Technology that requires user interaction often processes data in real-time to allow the user a seamless experience, when interacting with the technology to avoid wait times.
Regarding claim 7, Ohyu and Kim fail to explicitly teach the processing device according to claim 6, wherein the one or more processors is configured to extract the textual and/or numerical information from the at least one item based on optical character recognition.
However, Cohen teaches the processing device (Medical Data Processing Device, Para. 0059) is according to claim 6, wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to extract the textual and/or numerical information from the at least one item (Medical Data, Para. 0059) based on optical character recognition. (Para. 0043 and 0095) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s processor altered by Kim to incorporate Cohen’s Optical Character Recognition. Since doing so would provide the benefit of configuring the processor to be able to process data that otherwise it wouldn’t have been able to. Medical data comprises a wide variety of forms such as text, image, and video. Hence, optical character recognition can be used to retrieve data in other forms that would otherwise have to be manually inputted.
Regarding claim 14, Ohyu and Kim fail to explicitly teach the processing device according to claim 13, wherein the one or more processors is further configured to determine a response to the user’s query based on analysing further input display data received subsequent to the input display data.
However, Cohen teaches the processing device (Medical Data Processing Device, Para. 0059) according to claim 13, wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is further configured to determine a response to the user’s query based on analysing further input display data received subsequent to the input display data. (A graphical overlay can be displayed to alert the user of an incorrect error in the data and the recommended correction, Para. 0141. To recommend a correction subsequent input display data would have to be retrieved to cross-compare what incorrect information was inserted to respond with a recommendation.) Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Processor altered by Kim to incorporate Cohen’s Response to a Query. Since doing so would provide the benefit of correcting errors found in the information in a quick and easy manner. Errors present in medical data can lead to serious consequences, such as confusing patient data or misdiagnosing. Hence, being able to quickly and efficiently correct errors that are prompted to the user is important.
Regarding claim 17, Ohyu and Kim fail to explicitly teach the processing device according to claim l,
wherein the at least one item includes a patient identification item for uniquely identifying a patient;
and wherein the one or more processors is configured to extract the patient identification item from the user interface data to determine at least one of the augmentation data or the at least one augmentation position.
However, Cohen teaches the processing device according to claim l,
wherein the at least one item includes a patient identification item for uniquely identifying a patient; (Patient ID, Para. 0122 and 0033)
and wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to extract the patient identification item (Patient ID) from the user interface data to determine at least one of the augmentation data (Patient Medical Records) or the at least one augmentation position. (Location in Medical UI of Medical Monitor and EMR Application, Para. 0033 and 0122)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus altered by Kim to incorporate Cohen’s Patient Identification Item. Since doing so would provide the benefit of uniquely tracking each patient’s medical record, as well as grouping all related patient information by tying it to a unique ID. This allows the apparatus to easily retrieve all data associated to a patient by using the unique ID. The method of using a unique ID to identify information about a user/patient is standard practice.
Regarding claim 18, Ohyu teaches the processing device according to claim 17, further comprising:
a communication circuitry communicatively couplable to an external data source; (Network Interface 15, Para. 0055-0056 and 0063, Fig. 1)
However, Ohyu and Kim fail to explicitly teach:
and wherein the one or more processors is configured to retrieve at least a part of the augmentation data from the external data source based on the extracted patient identification item.
Cohen teaches:
and wherein the one or more processors (Hardware Processors, Para. 0031 and 0033) is configured to retrieve at least a part of the augmentation data (Medical data) from the external data source (EMR Application) based on the extracted patient identification item. (Patient ID are compared to identify medical data correspond to the monitor UI, Para. 0033 and 0122)
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ohyu’s Medical Data Processing Apparatus altered by Kim to incorporate Cohen’s Patient Identification Item. Since doing so would provide the benefit of uniquely tracking each patient’s medical record, as well as grouping all related patient information by tying it to a unique ID. This allows the apparatus to easily retrieve all data associated to a patient by using the unique ID. The method of using a unique ID to identify information about a user/patient is standard practice.
Regarding claim 19, Ohyu teaches the processing device (Medical Data Processing Apparatus, Para. 0055-0056) according to claim 18, wherein the augmentation data retrieved from the external data source ((Network Interface 15, Para. 0055-0056 and 0063, Fig. 1) includes medical data (Medical Images, Para. 0060) associated with the patient.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIANNA R COCHRAN whose telephone number is (571)272-4671. The examiner can normally be reached Mon-Fri. 7:30am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIANNA RENAE COCHRAN/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights