DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page 9, filed 12/31/2025, with respect to the specification objection have been fully considered and are persuasive. The objection of the specification has been withdrawn.
Applicant’s arguments, see page 9, filed 12/31/2025, with respect to the claim objections have been fully considered and are persuasive. The objections of the claims have been withdrawn.
Applicant’s arguments with respect to claim(s) 1-19 and 22 have been considered but are moot because the new ground of rejection does not rely on all references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The remarks state that the applied references do not perform the feature of the introduced claim amendment. The reference of Uehara is applied to cure the deficiencies of the previously applied references and will be briefly explained below.
Regarding the previously applied references, the primary reference performs the identification of a change area where image quality in a video is changed based on subject-related information and changing the image quality of the identified change area. This is detailed in ¶ [69]-[74] of the primary reference of Kuwata. The newly applied reference of Uehara is used to show both the patient and the doctor of selected area on the patient body in order to show the patient of the affected body part. This allows the user to move their affected body part in the center of the screen for better view with the patient able to see the emphasized body part. This is taught in ¶ [51]-[56] in the reference of Uehara. Therefore, this feature taught by Uehara, in combination with the rest of the features taught by the previously applied references, performs the features of the independent claims.
Thus, based on the above, the features of the claims are disclosed below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: input device in claims 7 and 17.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 8-13, 18, 19 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kuwata (US Pub 2020/0106821) in view of Nihei (JP Pub 2020-150512 (Pub Date: 9/17/2020)) and Uehara (JP Pub H0975404 (Pub Date: 3/25/1997)).
AMENDMENT TO CLAIMS
Re [Claim 1] (Currently Amended) Kuwata discloses an information processing system comprising:
at least one memory storing instructions, and at least one processor (e.g. the invention discloses a CPU and memory used to store instructions executed, which is taught in ¶ [31] and [32].) configured to execute the instructions to;
[0031] FIG. 3 is a diagram illustrating a hardware configuration of the IWB 100, according to an embodiment of the present invention. As illustrated in FIG. 3, the IWB 100 includes the camera 101, the touch panel display 102, the microphone 103, and the loudspeaker 104 that have been described in FIG. 2, and the IWB 100 further includes a system control 105 having a CPU (Central Processing Unit), auxiliary storage 106, memory 107, a communication I/F 108, an operation unit 109, and a recording device 110.
[0032] The system control 105 executes various programs stored in the auxiliary storage 106 or the memory 107 to perform various controls of the IWB 100. For example, the system control 105 includes a CPU, interfaces with peripheral units, a data access adjustment function, and the like. The system control 105 controls various types of hardware included in the IWB 100 to perform execution controls of various functions relating to a video conference provided by the IWB 100 (see FIG. 4).
acquire video data including a subject receiving a service and subject-related information related to the subject included in the video data (e.g. a video is acquired that includes a subject or a person. The system receives a detection of a user’s face or a user’s voice, which is taught in ¶ [62], [63], [69] and [70].);
[0062] Upon starting of the video conference, the video acquisition unit 122 acquires video data (YUV data) from the camera 101, and the voice acquisition unit 138 acquires voice data from the microphone 103 in step S504. In step S505, the video processor 150 performs video processing (described in detail in FIG. 6) on the video data acquired in step S504, and the voice processor 140 performs various voice processing on the voice data acquired in step S504. In step S506, the encoder 128 encodes the video data that has been video-processed in step S505. In step S507, the transmitter 130 transmits the video data encoded in step S506 to an external apparatus such as another IWB 100 through a network 16 together with the voice data acquired in step S504.
[0063] In parallel with steps S504 to S507, the receiver 132 receives the video data and voice data transmitted from another IWB 100 through the network 16 in step S508. The decoder 134 decodes the video data received in step S508. In step S510, the voice processor 140 performs various types of voice processing on the voice data received in step S508. In step S511, the display controller 136 displays a video on the touch panel display 102 based on the video data decoded in step S509, and the voice output unit 142 outputs a voice from the loudspeaker 104 based on the voice data that has been voice-processed in step S510. In step S511, the display controller 136 may further display a video (i.e., a video at the location of the IWB itself) on the touch panel display 102, based on the video data acquired in step S504.
[0068] In step S604, with respect to the single frame image selected in step S601, the image quality determination unit 153 sets an image quality for each of the blocks divided in step S602 based on an analysis result of the high frequency components obtained in step S603 so as to generate an image quality level map.
[0069] Next, in step S605, the specific area detector 154 detects one or more of face areas where a face of a person is displayed in the single frame image selected in step S601. Further, in step S606, the specific area detector 154 detects a speaker's area where a face of a person who converses is displayed, from among the face areas detected in step S605.
[0070] In step S607, the image quality determination unit 153 changes the image quality level map generated in step S604, based on the detection result of the face area in step S605 and the detection result of the speaker's area in step S606. For example, in the image quality level map generated in step S604, the image quality determination unit 153 changes the image quality of a face area that is a speaker's area to “A (highest image quality)”, and also changes the image quality of a face area that is not a speaker's area to “B (high image quality)”. In addition, with respect to the image quality level map generated in step S604, the image quality determination unit 153 changes an image quality of an area that is not a peripheral area around the speaker's area to “D (low image quality)” without changing an image quality of the peripheral area around the speaker's area.
identify a change area where image quality in the video data is changed based on the subject-related information (e.g. the system determines the face and speaker’s face area as an area to change. The voice and face are considered as subject-related information that is used to identify the area, which is taught in ¶ [69] and [70] above.); and
change the image quality in the identified change area (e.g. the invention discloses changing the image quality of an area that is identified based on identifying different areas associated with a user’s voice and face, which is taught in ¶ [71]-[74].).
[0071] Next, in step S608, the image quality determination unit 153 determines whether a network bandwidth used for a video conference has extra capacity. In step S609, when the image quality determination unit 153 determines that a network bandwidth has extra capacity (step S608: Yes), the image quality determination unit 153 changes the image quality level map to improve an image quality of a part of the areas. For example, the image quality determination unit 153 may change an image quality of the face area that is not the speaker's area from “B (high image quality) to “A (highest image quality)”, and may return an image quality of an area that is not the peripheral area around the speaker's area to the image quality set in the image quality level map originally generated in step S604. Then, the video processor 150 progresses the processing to step S612.
[0072] Meanwhile, in step S610, when the image quality determination unit 153 determines that a network bandwidth used for a video conference does not have extra capacity (step S608: No), the image quality determination unit 153 determines whether the network bandwidth is short of capacity. When the image quality determination unit 153 determines that the network bandwidth is short of capacity (step S610: Yes), the image quality determination unit 153 changes an image quality of other areas excluding the face area to “D (low image quality)” in step S611. Then, the video processor 150 progresses the processing to step S612.
[0073] Meanwhile, in step S610, when the image quality determination unit 153 determines that a network bandwidth is not short of capacity (step S610: No), the video processor 150 progresses the processing to step S612.
[0074] In step S612, the image quality adjuster 155 adjusts an image quality, pixel by pixel, with respect to the frame image selected in step S601, according to the final image quality level map.
However, Kuwata fails to specifically teach the feature of acquire video data including a subject receiving a medical service.
However, this is well known in the art as evidenced by Nihei. Similar to the primary reference, Nihei discloses acquiring video of a patient by a doctor (same field of endeavor or reasonably pertinent to the problem).
Nihei discloses acquire video data including a subject receiving a medical service (e.g. the invention discloses a doctor receiving a video of a patient for medical care, which is taught in ¶ [65] and [66].).
[0065]If the installation location of the media receiving apparatus 502 is changed, it can be used for purposes other than monitoring. For example, when the media receiving apparatus 502 is a video display program that operates on a server or a PC installed in a hospital, the media receiving apparatus 200 can be used for remote medical care or the like. In the case of remote medical care, a doctor in a hospital can increase the image quality of only an affected part in a video of a patient at home.[0066]As described above, the media receiving apparatus 502 notifies the media coding apparatus 504 of the request condition, and the media coding apparatus 504 distributes the video while dynamically changing the coding parameter according to the request condition. As described above, the media coding apparatus 504 according to the first and second embodiments of the present invention can minimize the data amount of the media stream while satisfying the required quality of a plurality of applications, and thus can transmit necessary data with necessary quality without pressing the band of the network 503. Therefore, even if the network 503 is charged on a pay-as-you-go basis, the usage fee can be suppressed.
Therefore, in view of Nihei, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of acquire video data including a subject receiving a medical service, incorporated in the device of Kuwata, in order to acquire video for remote medical care and moderate image quality of the video, which transmit necessary data while reducing the resource use of the network (as stated in Nihei ¶ [66]).
However, the combination above fails to specifically teach the features of notify the subject of a body part corresponding to the change area so that the subject can recognize the body part.
However, this is well known in the art as evidenced by Uehara. Similar to the primary reference, Uehara discloses focusing a display on an area of a patient (same field of endeavor or reasonably pertinent to the problem).
Uehara discloses notify the subject of a body part corresponding to the change area so that the subject can recognize the body part (e.g. the previously applied references disclose changing an area of an affected part of the body being examined by a doctor in a telemedicine setting. The changed area is modified by improving the image quality of the affected area. The Uehara reference discloses seeing this impacted area on a patient on the doctor’s monitor, and also showing what the doctor sees on the patient monitor. This is used to show the affected area the doctor sees to the patient, which is taught in ¶ [51]-[56].).
[0051] In the case of this telemedicine system, the image of the patient is taken by the camera 24 on the patient side. The image captured here is input to the communication I / F 44 of the monitoring device 3 through the communication I / F 31 and the external communication network 1, and is displayed on the display device 38 through this. The doctor observes the appearance of the patient displayed on the display device 38, and as shown in FIG. 2, directly traces around the affected area 50 displayed on the screen 38a of the display device 38 with the light pen 42 and touches it. When a surrounding figure is drawn, the figure in the selected area 49 is displayed in a rectangular shape or the like so as to be superimposed on the affected area 50 on the screen 38a. The figure drawn here may be a rectangle, a circle, or any other closed curve if it is a figure that can surround the affected area and emphasize the affected area.
[0052] The image on the patient side shown in FIG. 2 may be displayed on the display device 29 on the patient side at the same time as the display device 38. In this case, since the patient himself / herself can see his / her own figure and the affected area, care can be taken to move the body so that the affected area 50 is located at the center of the screen 38a.
[0053] Here, if the position of the selection area (rectangular figure) 49 indicating the affected area 50 is fixed on the screen 38a, as shown in FIG. Then, the affected part 50 is separated.
[0054]In this case, the patient has the affected area 50 in the selected area 4.
Although the body may be moved so as to enter inside 9, when the image of the affected area 50 is unclear or the selected area 49 itself showing the affected area 50 is small, the affected area 50 is indicated in the figure of the selected area 49.
It becomes difficult to keep it.
[0055] Therefore, as shown in FIG. 4, even if the patient moves, it is necessary to always recognize and track the position of the affected area contained in the selected area 49 and move the selected area 49 simultaneously with the affected area 50. is there.
[0056] Here, it is necessary to always extract and track the same image in the selection area 49 set as the initial state even if the patient moves, from the entire screen.
Therefore, in view of Uehara, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of notify the subject of a body part corresponding to the change area so that the subject can recognize the body part, incorporated in the device of Kuwata, as modified by Nihei, in order to display to the patient the image of the affected body part, which can aid in arranging the body part in the area of the screen to be seen easier and increase ease of use of the telemedicine system (as stated in Uehara ¶ [25] and [26]).
Re [Claim 2] (Currently Amended) Kuwata discloses the information processing system according to claim 1, wherein the at least one processor is further configured to execute the instructions to:
output a plurality of candidates for the change area based on the subject-related information (e.g. the system divides a frame into several blocks and determines areas of a face and speaker’s area. These areas are determined as candidate areas for change, which is taught in ¶ [68]-[70] above.), and
accept a selection of at least one candidate from the plurality of candidates (e.g. the system accepts as a selection the areas of the speaker associated with a face as a candidate area to be changed, which is taught in ¶ [68]-[70] above. Also, depending on the network bandwidth capacity, the areas other than the face can be output as a selection for changing, which is taught in ¶ [71]-[74] above.),
wherein the image quality in an area corresponding to the selected candidate is changed (e.g. the image quality is changed associated with the face of the speaker as well as the areas that are not associated with the face based on the bandwidth of the network, which is taught in ¶ [68]-[74] above.).
Re [Claim 3] (Currently Amended) Kuwata discloses the information processing system according to claim 1, wherein the subject-related information is at least one of medical record information of the subject, medical interview information of the subject, voice information of the subject included in the video data, or motion information of the subject included in the video data (e.g. the invention discloses voice information of a subject included in the video, which is taught in ¶ [62] and [63] above.).
[Claim 8] (Currently Amended) Kuwata discloses the information processing system according to claim 1, further comprising a first terminal and a second terminal configured to communicate bi-directionally (e.g. there are multiple IWBs that can communicate within the video conference, which is taught in ¶ [63] above.),
wherein the first terminal configured to execute the instructions to image the subject and generating the video data (e.g. the first terminal can be the recording terminal device that has a user that is speaking. This will cause the system to record this person and send it to the other individuals in the video conference, which is taught in ¶ [62] and [63] above.), and
wherein the second terminal configured to execute the instructions to display a video wherein the image quality in the change area has been changed in the video data generated by the first terminal (e.g. the receiving computing devices can receive the video from the speaker’s computer, which is taught in ¶ [62] and [63] above. The image can reflect parts of the image that has higher quality than others based on the speaker and the network bandwidth capacity, which is taught in ¶ [68]-[74] above.).
[Claim 9] (Currently Amended) However, Kuwata fails to specifically teach the feature of the information processing system according to claim 8, wherein compression ratio of the generated video data is controlled to change the image quality in the change area.
However, this is well known in the art as evidenced by Nihei. Similar to the primary reference, Nihei discloses acquiring video of a patient by a doctor (same field of endeavor or reasonably pertinent to the problem).
Nihei discloses wherein compression ratio of the generated video data is controlled to change the image quality in the change area (e.g. the system discloses in the background of having a high-quality compression to areas or regions desired while keeping a minimum image quality of other parts of the screen constant, which is taught in ¶ [03].).
[0003]Also, scalable coding (hierarchical coding) is known as a recent representative example of coding technology. In today's network environment, a high-speed communication path such as an optical fiber and a relatively low-speed wireless communication path such as a mobile terminal are mixed. Hierarchical coding enables efficient coding according to the performance of the receiving side in such a network environment. An example of a video distribution system using such hierarchical coding is described in Patent Document 2. According to Patent Document 2, by controlling hierarchical encoding, it is possible to perform high-quality compression on a region desired by a user while maintaining the minimum image quality of the entire screen constant.
Therefore, in view of Nihei, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of wherein compression ratio of the generated video data is controlled to change the image quality in the change area, incorporated in the device of Kuwata, in order to contain a change area with a certain amount of compression, which can allow for efficient compression coding (as stated in Nihei ¶ [03]).
[Claim 10] (Currently Amended) Kuwata discloses the information processing system according to claim 8, wherein the first terminal is further configured to execute the instructions to notify the subject of the change area. (e.g. the first computing device that performs the change in area based on face and speaking along with network bandwidth is notified to the own system’s display to show the area change, which is taught in ¶ [64] and [68]-[74] above.)
[Claim 11] (Currently Amended) Kuwata discloses an information processing device comprising:
at least one memory storing instructions, and at least one processor (e.g. the invention discloses a CPU and memory used to store instructions executed, which is taught in ¶ [31] and [32] above.) configured to execute the instructions to:
acquire video data including a subject receiving a service and subject-related information related to the subject included in the video data (e.g. a video is acquired that includes a subject or a person. The system receives a detection of a user’s face or a user’s voice, which is taught in ¶ [62], [63], [69] and [70] above.);
identify a change area where image quality in the video data is changed based on the subject-related information (e.g. the system determines the face and speaker’s face area as an area to change. The voice and face are considered as subject-related information that is used to identify the area, which is taught in ¶ [69] and [70] above.);
output an image quality change instruction to change the image quality in the identified change area (e.g. the invention discloses changing the image quality of an area that is identified as the based on identifying different areas associated with a user’s voice and face, which is taught in ¶ [71]-[74] above.).
However, Kuwata fails to specifically teach the feature of acquire video data including a subject receiving a medical service.
However, this is well known in the art as evidenced by Nihei. Similar to the primary reference, Nihei discloses acquiring video of a patient by a doctor (same field of endeavor or reasonably pertinent to the problem).
Nihei discloses acquire video data including a subject receiving a medical service (e.g. the invention discloses a doctor receiving a video of a patient for medical care, which is taught in ¶ [65] and [66] above.).
Therefore, in view of Nihei, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of acquire video data including a subject receiving a medical service, incorporated in the device of Kuwata, in order to acquire video for remote medical care and moderate image quality of the video, which transmit necessary data while reducing the resource use of the network (as stated in Nihei ¶ [66]).
However, the combination above fails to specifically teach the features of and notify the subject of a body part corresponding to the change area so that the subject can recognize the body part.
However, this is well known in the art as evidenced by Uehara. Similar to the primary reference, Uehara discloses focusing a display on an area of a patient (same field of endeavor or reasonably pertinent to the problem).
Uehara discloses and notify the subject of a body part corresponding to the change area so that the subject can recognize the body part (e.g. the previously applied references disclose changing an area of an affected part of the body being examined by a doctor in a telemedicine setting. The changed area is modified by improving the image quality of the affected area. The Uehara reference discloses seeing this impacted area on a patient on the doctor’s monitor, and also showing what the doctor sees on the patient monitor. This is used to show the affected area the doctor sees to the patient, which is taught in ¶ [51]-[56] above.).
Therefore, in view of Uehara, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of and notify the subject of a body part corresponding to the change area so that the subject can recognize the body part, incorporated in the device of Kuwata, as modified by Nihei, in order to display to the patient the image of the affected body part, which can aid in arranging the body part in the area of the screen to be seen easier and increase ease of use of the telemedicine system (as stated in Uehara ¶ [25] and [26]).
[Claim 12] (Currently Amended) Kuwata discloses the information processing device according to claim 11, wherein the at least one processor is further configured to execute the instructions to:
output a plurality of candidates for the change area based on the subject-related information (e.g. the system divides a frame into several blocks and determines areas of a face and speaker’s area. These areas are determined as candidate areas for change, which is taught in ¶ [68]-[70] above.), and
accept a selection of at least one candidate from the plurality of candidates (e.g. the system accepts as a selection the areas of the speaker associated with a face as a candidate area to be changed, which is taught in ¶ [68]-[70] above. Also, depending on the network bandwidth capacity, the areas other than the face can be output as a selection for changing, which is taught in ¶ [71]-[74] above.),
wherein the image quality change instruction is output to change the image quality in an area corresponding to the selected candidate (e.g. the image quality is changed associated with the face of the speaker as well as the areas that are not associated with the face based on the bandwidth of the network, which is taught in ¶ [68]-[74] above.).
[Claim 13] (Currently Amended) Kuwata discloses the information processing device according to claim 11, wherein the subject-related information is at least one of medical record information of the subject, medical interview information of the subject, voice information of the subject included in the video data, or motion information of the subject included in the video data (e.g. the invention discloses voice information of a subject included in the video, which is taught in ¶ [62] and [63] above.).
[Claim 18] (Currently Amended) The information processing device according to claim 11, wherein the at least one processor is further configured to execute the instructions to:
acquire, via a network, the video data generated by shooting the subject by an video camera (e.g. the first terminal can be the recording terminal device that has a user that is speaking. This will cause the system to record this person and send it to the other individuals in the video conference, which is taught in ¶ [62] and [63] above.), and
display the video in which the image quality in the change area has been changed, in the video data generated by the video camera, in accordance with the image quality change instruction (e.g. the receiving computing devices can receive the video from the speaker’s computer, which is taught in ¶ [62] and [63] above. The image can reflect parts of the image that has higher quality than others based on the speaker and the network bandwidth capacity, which is taught in ¶ [68]-[74] above.).
[Claim 19] (Currently Amended) However, Kuwata fails to specifically teach the features of the information processing device according to claim 18, wherein the at least one processor is further configured to execute the instructions to:
control a compression ratio of the video data generated by the video camera and generate the image quality change instruction to change the image quality in the change area.
However, this is well known in the art as evidenced by Nihei. Similar to the primary reference, Nihei discloses acquiring video of a patient by a doctor (same field of endeavor or reasonably pertinent to the problem).
Nihei discloses control a compression ratio of the video data generated by the video shooting device and generate the image quality change instruction to change the image quality in the change area (e.g. the system discloses in the background of having a high-quality compression to areas or regions desired while keeping a minimum image quality of other parts of the screen constant, which is taught in ¶ [03] above.).
Therefore, in view of Nihei, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of control a compression ratio of the video data generated by the video shooting device and generate the image quality change instruction to change the image quality in the change area, incorporated in the device of Kuwata, in order to contain a change area with a certain amount of compression, which can allow for efficient compression coding (as stated in Nihei ¶ [03]).
[Claim 20 - 21] (Canceled)
[Claim 22] (Original) /Kuwata discloses a non-transitory computer-readable medium storing a program for causing a computer to execute:
processing of acquiring video data including a subject receiving a medical service and subject-related information related to the subject included in the video data (e.g. a video is acquired that includes a subject or a person. The system receives a detection of a user’s face or a user’s voice, which is taught in ¶ [62], [63], [69] and [70] above.);
processing of identifying a change area where image quality in the video data is changed based on the subject-related information (e.g. the system determines the face and speaker’s face area as an area to change. The voice and face are considered as subject-related information that is used to identify the area, which is taught in ¶ [69] and [70] above.); and
processing of changing the image quality in the identified change area (e.g. the invention discloses changing the image quality of an area that is identified as the based on identifying different areas associated with a user’s voice and face, which is taught in ¶ [71]-[74] above.).
However, Kuwata fails to specifically teach the feature of processing of acquiring video data including a subject receiving a medical service.
However, this is well known in the art as evidenced by Nihei. Similar to the primary reference, Nihei discloses acquiring video of a patient by a doctor (same field of endeavor or reasonably pertinent to the problem).
Nihei discloses processing of acquiring video data including a subject receiving a medical service (e.g. the invention discloses a doctor receiving a video of a patient for medical care, which is taught in ¶ [65] and [66] above.).
Therefore, in view of Nihei, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of processing of acquiring video data including a subject receiving a medical service, incorporated in the device of Kuwata, in order to acquire video for remote medical care and moderate image quality of the video, which transmit necessary data while reducing the resource use of the network (as stated in Nihei ¶ [66]).
However, the combination above fails to specifically teach the features of and processing of notifying the subject of a body part corresponding to the change area so that the subject can recognize the body part.
However, this is well known in the art as evidenced by Uehara. Similar to the primary reference, Uehara discloses focusing a display on an area of a patient (same field of endeavor or reasonably pertinent to the problem).
Uehara discloses and processing of notifying the subject of a body part corresponding to the change area so that the subject can recognize the body part (e.g. the previously applied references disclose changing an area of an affected part of the body being examined by a doctor in a telemedicine setting. The changed area is modified by improving the image quality of the affected area. The Uehara reference discloses seeing this impacted area on a patient on the doctor’s monitor, and also showing what the doctor sees on the patient monitor. This is used to show the affected area the doctor sees to the patient, which is taught in ¶ [51]-[56] above.).
Therefore, in view of Uehara, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of and processing of notifying the subject of a body part corresponding to the change area so that the subject can recognize the body part, incorporated in the device of Kuwata, as modified by Nihei, in order to display to the patient the image of the affected body part, which can aid in arranging the body part in the area of the screen to be seen easier and increase ease of use of the telemedicine system (as stated in Uehara ¶ [25] and [26]).
Claim(s) 4, 5, 7, 14, 15 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kuwata, as modified by Nihei and Uehara, as applied to claims 1 and 11 above, and further in view of Sugiyama (JP Pub 2010-187711 (Pub Date: 9/2/2010)).
[Claim 4] (Currently Amended) Kuwata discloses the information processing system according to claim 3, wherein the at least one processor is further configured to execute the instructions to:
and
identify the change area based on at least one of the subject-related information or the voice data (e.g. the system determines the face and speaker’s face area as an area to change. The voice and face are considered as subject-related information that is used to identify the area, which is taught in ¶ [69] and [70] above.).
However, Kuwata fails to specifically teach the feature of acquire voice data of a provider of a medical service.
However, this is well known in the art as evidenced by Sugiyama. Similar to the primary reference, Sugiyama discloses communication between doctor and patient (same field of endeavor or reasonably pertinent to the problem).
Sugiyama discloses acquire voice data of a provider of a medical service (e.g. the doctor can communicate to a patient with a communication terminal and voice input, which is taught in ¶ [16].).
[0016] In the communication terminal 200 on the doctor side, the control unit 21, the command input unit 22, the voice input / output unit 23, the image display unit 24, the camera 25, the camera control unit 26, and the communication unit 27 have the same configurations as those indicated by 11 to 17 of the communication terminal 100, respectively. The communication terminal 200 on the doctor side further includes an image data storage unit 28, a reference region extraction unit 29, and an imaging region determination unit 30. The image data storage unit 28 stores the whole body image of the subject, the position information of the face portion, the affected part imaging data, and the like sent from the communication terminal 100. The reference region extraction unit automatically extracts a reference region (a face region in the present embodiment) from a desired video sent from the communication terminal 100. The imaging region determination unit 30 determines an imaging target region (affected part imaging region) for the second and subsequent imaging operations with reference to the reference region (face region).
Therefore, in view of Sugiyama, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of acquire voice data of a provider of a medical service, incorporated in the device of Kuwata, as modified by Nihei and Uehara, in order to communicate with a patient to perform a remote diagnosis when capturing an affected area of a patient, which can improve the accuracy of diagnosis (as stated in Sugiyama ¶ [07]).
[Claim 5] (Currently Amended) The information processing system according to claim 4, wherein, when the voice information of the subject and the voice data of the provider are acquired, priority is given to the voice data of the provider (e.g. if voice data of a provider of the video conference is occurs after the voice data of an initial user, the video and voice of the last speaker, which can be the provider of the video conference, is given priority in showing the video associated with the voice. This is taught in ¶ [63] above.) and
the change area is identified based on the voice data (e.g. the system determines the face and speaker’s face area as an area to change. The voice and face are considered as subject-related information that is used to identify the area, which is taught in ¶ [69] and [70] above.).
Re [Claim 7] (Currently Amended) However, Kuwata fails to specifically teach the features of the information processing system according to claim 1, further comprising an input device configured to imitate a human body,
wherein the subject-related information includes specified position information on the human body inputted from the input device means.
However, this is well known in the art as evidenced by Sugiyama. Similar to the primary reference, Sugiyama discloses communication between doctor and patient (same field of endeavor or reasonably pertinent to the problem).
Sugiyama discloses further comprising an input device (interpretation: an input means that imitates the human body (hereinafter referred to as a "human body model input device"), including three-dimensional human body models such as human models and dolls, and two-dimensional human body models such as illustrations of the human body, can be used as input devices that accept doctor's input operations, which is taught in ¶ [35]. In the input information display area 113, a diagram imitating the human body is displayed. When the specified position information is generated by the human body model input device described above, marks, icons, and others displayed on the part corresponding to the specified position information in the diagram of the human body. That is, the input information display area 113 displays a clear indication of the patient's body part to be visually examined, which the doctor presses down on the human body model input device, which is taught in ¶ [44]. These interpretations and its equivalents are utilized for this claim term hereinafter in the Office Action.) configured to imitate a human body (e.g. the system can input the whole human body of a patient, which is taught in ¶ [16].),
[0016]In the communication terminal 200 on the doctor side, the control unit 21, the command input unit 22, the voice input / output unit 23, the image display unit 24, the camera 25, the camera control unit 26, and the communication unit 27 have the same configurations as those indicated by 11 to 17 of the communication terminal 100, respectively. The communication terminal 200 on the doctor side further includes an image data storage unit 28, a reference region extraction unit 29, and an imaging region determination unit 30. The image data storage unit 28 stores the whole body image of the subject, the position information of the face portion, the affected part imaging data, and the like sent from the communication terminal 100. The reference region extraction unit automatically extracts a reference region (a face region in the present embodiment) from a desired video sent from the communication terminal 100. The imaging region determination unit 30 determines an imaging target region (affected part imaging region) for the second and subsequent imaging operations with reference to the reference region (face region).
wherein the subject-related information includes specified position information on the human body inputted from the input device (e.g. the invention discloses specific affected regions on a whole body image that can be identified before these areas are associated with and shown on the whole body, which is taught in ¶ [25]-[29].).
[0025]<Step S203>Next, the doctor performs a process of setting the affected region to be imaged on the moving image of the real moving image display unit 303 on the doctor-side GUI. Specifically, after selecting the region designation button 308 in FIG. 3, the doctor (user) performs a region designation operation such as designating two points on the real moving image display portion 303 with a mouse or the like. The control unit 21 sets a rectangular region 313 including the two designated points as an imaging region (region of interest) of the affected part A.[0026]<Step S204>When the imaging region is set in step S203 and the affected part imaging start button 309 is selected, the reference region extraction unit 29 extracts one frame from the whole-body image captured at the timing when the imaging region is selected. Then, it is stored in the image data storage unit 28 as a whole body image (still image) at the time of the first photographing. In addition, the PTZ information of pan, tilt, and zoom of the camera 15 at the time of capturing the one frame is also extracted and stored in the image data storage unit 28. The process in step S204 described above corresponds to step S1608,S1609 .[0027]<Step S205>Next, the reference region extraction unit 29 automatically extracts a region of a specific image (face) as a reference region from the still image of the entire image extracted in step S204, and obtains position information of the face region (corresponding to S1609). Since the method of extracting the face region is a known image processing technique, it will not be described in detail here.[0028]<Step S206>Here, the still image and the PTZ information stored in the image-data storage unit 28 in the step S204, and the image and the position (indicating the position and the size) of the face region extracted in the step S205 are registered in a database as shown in FIG. 13. Further, the position (indicating the position and the size) of the imaging area (rectangular area) of the affected part designated in the step S203 is registered in the database in association with the entire image and the face area. In this way, the initial whole body image data, the initial face data, and the position information in the initial affected part data illustrated in FIG. 13 are registered. That is, the database stores, as one record, region-of-interest information indicating the position and size of a reference region (for example, a face region) indicating the region of a specific image extracted from a captured image obtained by the camera 15, and the position and size of a region of interest (imaging region) designated in the captured image in the captured image. Note that the database is formed in the image data storage unit 28.[0029]<Step S207>The control unit 21 displays the still image of the whole body image on the initial whole body image display unit 301, and graphically displays the rectangle of the imaging region. FIG. 5 is a diagram illustrating a display example on the first time whole body image display unit 301. Here, the displayed rectangle of the imaging region (the affected region set in the step S203) may be adjusted on the still picture. In addition, in the above procedure, the affected region is set in the real moving image display unit 303, but the present invention is not limited thereto. For example, a still image extraction instruction button may be provided on the user interface in FIG. 3, and the affected region may be set on the extracted still image. In this case, when the pressing of the still image extraction instruction button is detected while the moving image is displayed on the real moving image display portion 303, the control unit 21 extracts the still image and displays the still image on the first time whole body image display portion 301. Then, when the region designation button 308 is pressed, the control unit 21 controls the apparatus so that the affected region can be set on the still image in the initial whole body image display portion 301.
Therefore, in view of Sugiyama, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of further comprising an input device configured to imitate a human body, wherein the subject-related information includes specified position information on the human body inputted from the input device means, incorporated in the device of Kuwata, in order to communicate with a patient to perform a remote diagnosis when capturing an affected area of a patient, which can improve the accuracy of diagnosis (as stated in Sugiyama ¶ [07]).
[Claim 14] (Currently Amended) Kuwata discloses the information processing device according to claim 13, wherein the at least one processor is further configured to execute the instructions to;
and
identify the change area based on at least one of the subject-related information or the voice data (e.g. the system determines the face and speaker’s face area as an area to change. The voice and face are considered as subject-related information that is used to identify the area, which is taught in ¶ [69] and [70] above.).
However, Kuwata fails to specifically teach the feature of acquire voice data of a provider of a medical service.
However, this is well known in the art as evidenced by Sugiyama. Similar to the primary reference, Sugiyama discloses communication between doctor and patient (same field of endeavor or reasonably pertinent to the problem).
Sugiyama discloses acquire voice data of a provider of a medical service (e.g. the doctor can communicate to a patient with a communication terminal and voice input, which is taught in ¶ [16] above.).
Therefore, in view of Sugiyama, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of acquire voice data of a provider of a medical service, incorporated in the device of Kuwata, as modified by Nihei and Uehara, in order to communicate with a patient to perform a remote diagnosis when capturing an affected area of a patient, which can improve the accuracy of diagnosis (as stated in Sugiyama ¶ [07]).
[Claim 15] (Currently Amended) The information processing device according to claim 14, wherein, when the voice information of the subject and the voice data of the provider are acquired, priority is given to the voice data of the provider (e.g. if voice data of a provider of the video conference is occurs after the voice data of an initial user, the video and voice of the last speaker, which can be the provider of the video conference, is given priority in showing the video associated with the voice. This is taught in ¶ [63] above.) and
the change area is identified based on the voice data (e.g. the system determines the face and speaker’s face area as an area to change. The voice and face are considered as subject-related information that is used to identify the area, which is taught in ¶ [69] and [70] above.).
[Claim 17] (Currently Amended) The information processing device according to claim 11, further comprising an input device configured to imitate a human body,
wherein the subject-related information includes specified position information on the human body inputted from the input device.
However, this is well known in the art as evidenced by Sugiyama. Similar to the primary reference, Sugiyama discloses communication between doctor and patient (same field of endeavor or reasonably pertinent to the problem).
Sugiyama discloses further comprising an input device configured to imitate a human body (e.g. the system can input the whole human body of a patient, which is taught in ¶ [16] above.),
wherein the subject-related information includes specified position information on the human body inputted from the input device (e.g. the invention discloses specific affected regions on a whole body image that can be identified before these areas are associated with and shown on the whole body, which is taught in ¶ [25]-[29] above.).
Therefore, in view of Sugiyama, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of further comprising an input device configured to imitate a human body, wherein the subject-related information includes specified position information on the human body inputted from the input device, incorporated in the device of Kuwata, in order to communicate with a patient to perform a remote diagnosis when capturing an affected area of a patient, which can improve the accuracy of diagnosis (as stated in Sugiyama ¶ [07]).
Claim(s) 6 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kuwata, as modified by Nihei and Uehara, as applied to claims 1 and 11 above, and further in view of Kuwata ‘462 (US Pub 2019/0306462).
[Claim 6] (Currently Amended) However, Kuwata fails to specifically teach the features of the information processing system according to claim 3, wherein, when the voice information and the motion information of the subject are acquired, the change area is identified based on both the voice information and the motion information.
However, this is well known in the art as evidenced by Kuwata ‘462. Similar to the primary reference, Kuwata ‘462 discloses detecting motion and determining an area image quality to change (same field of endeavor or reasonably pertinent to the problem).
Kuwata ‘462 discloses wherein, when the voice information and the motion information of the subject are acquired, the change area is identified based on both the voice information and the motion information (e.g. in the primary reference, a speaker and the face is detected in order to determine an area to change. The reference of Kuwata ‘462 discloses using the face and the motion of a user in order to determine an area to change the image quality. Combining these references together to find the speaker, face and motion of a user to change an area of image quality performs the features of the invention above. The motion and face detection is explained in ¶ [62]-[67].).
[0062] First, the video processing unit 124 selects one frame image from among a plurality of frame images constituting video data in order from oldest to newest (step S601). Next, the motion region detecting unit 126A detects one or more motion regions, each of which is a region in which motion of an object is detected, from the one frame image selected in step S601 (step S602). The face region detecting unit 126B detects one or more face regions, each of which is a region in which the face of an object is detected, from the one piece of video data, which is obtained by the video obtaining unit 122 (step S603). At this time, the face region detecting unit 126B may determine a region in which a face is detected over a predetermined number of successive frame images to be a face region in order to prevent erroneous detection.
[0063] Then, the video processing unit 124 sets, on the basis of the result of detection of the one or more face regions in step S603, the low-image-quality region, the medium-image-quality region, and the high-image-quality region for the one frame image selected in step S601 (step S604). Specifically, the video processing unit 124 sets each face region as the high-image-quality region. The video processing unit 124 sets a region other than the one or more face regions as the low-image-quality region. The video processing unit 124 sets the boundary part between the high-image-quality region and the low-image-quality region as the medium-image-quality region.
[0064] Subsequently, the video processing unit 124 determines whether the low-image-quality region (that is, the region in which no face is detected) set in step S604 includes a region that has just been a face region (step S605). For example, the video processing unit 124 stores the result of detecting one or more face regions in the previous frame image in the memory 107 and refers to the detection result to thereby determine whether a region that has just been a face region is included.
[0065] If it is determined in step S605 that a region that has just been a face region is not included (No in step S605), the video processing unit 124 advances the processing to step S608. On the other hand, if it is determined in step S605 that a region that has just been a face region is included (Yes in step S605), the video processing unit 124 determines whether the region that has just been a face region corresponds to one of the motion regions detected in step S602 (step S606).
[0066] If it is determined in step S606 that the region that has just been a face region does not correspond to any of the motion regions detected in step S602 (No in step S606), the video processing unit 124 advances the processing to step S608. On the other hand, if it is determined in step S606 that the region that has just been a face region corresponds to one of the motion regions detected in step S602 (Yes in step S606), the video processing unit 124 resets the region as the high-image-quality region (step S607). This is because the region is highly likely a region in which a face is present but is not detected because, for example, the orientation of the face changes. At the same time, the video processing unit 124 resets the boundary part between the region and the low-image-quality region as the medium-image-quality region. Then, the video processing unit 124 advances the processing to step S608.
[0067] In step S608, the video processing unit 124 makes an image-quality adjustment for each of the regions set as the low-image-quality region, the medium-image-quality region, and the high-image-quality region in step S604 and S607 so as to have corresponding image qualities. For example, the video processing unit 124 maintains the original image quality of the region set as the high-image-quality region. For the regions set as the medium-image-quality region and the low-image-quality region, the video processing unit 124 uses some publicly known image-quality adjustment method (for example, a resolution adjustment, a contrast adjustment, low-pass filtering application, a frame rate adjustment, etc.) to decrease the image quality of each of the regions from the original image quality thereof so that the region set as the medium-image-quality region has a medium image quality and the region set as the low-image-quality region has a low image quality. At this time, the video processing unit 124 makes the boundary part set as the medium-image-quality region have a medium image quality such that the image quality of the boundary part decreases toward the region set as the low-image-quality region in a stepwise manner. Accordingly, the difference in image quality between the high-image-quality region and the low-image-quality region can be made less noticeable.
Therefore, in view of Kuwata ‘462, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of wherein, when the voice information and the motion information of the subject are acquired, the change area is identified based on both the voice information and the motion information, incorporated in the device of Kuwata, as modified by Nihei and Uehara, in order to use motion with another factor when determining an area to change in image quality, which can reduce the data amount transferred and make differences in image quality less noticeable (as stated in Kuwata ‘462 ¶ [22]).
[Claim 16] (Currently Amended) However, Kuwata fails to specifically teach the features of the information processing device according to claim 13, wherein, when the voice information and the motion information of the subject are acquired, the change area is identified based on both the voice information and the motion information.
However, this is well known in the art as evidenced by Kuwata ‘462. Similar to the primary reference, Kuwata ‘462 discloses detecting motion and determining an area image quality to change (same field of endeavor or reasonably pertinent to the problem).
Kuwata ‘462 discloses wherein, when the voice information and the motion information of the subject are acquired, the change area is identified based on both the voice information and the motion information (e.g. in the primary reference, a speaker and the face is detected in order to determine an area to change. The reference of Kuwata ‘462 discloses using the face and the motion of a user in order to determine an area to change the image quality. Combining these references together to find the speaker, face and motion of a user to change an area of image quality performs the features of the invention above. The motion and face detection is explained in ¶ [62]-[67] above.).
Therefore, in view of Kuwata ‘462, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of wherein, when the voice information and the motion information of the subject are acquired, the change area is identified based on both the voice information and the motion information, incorporated in the device of Kuwata, as modified by Nihei and Uehara, in order to use motion with another factor when determining an area to change in image quality, which can reduce the data amount transferred and make differences in image quality less noticeable (as stated in Kuwata ‘462 ¶ [22]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kanda sets a display condition based on a region of interest.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAD S DICKERSON whose telephone number is (571)270-1351. The examiner can normally be reached Monday-Friday 10AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at 571-270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHAD DICKERSON/ Primary Examiner, Art Unit 2682