DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1, 3-5, 7-11, 22, and 24-37 are pending for examination in the application filed 06/29/2026. Claims 1, 22, 31, and 33 are currently amended.
Response to Arguments and Amendments
Applicant’s arguments regarding the amendments to independent claims 1, 22, 31, and 33 on pages 10-11 of the Remarks filed 06/29/2026 have been considered but are moot because the new ground of rejection does not rely on the references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument, as facilitated by the newly added amendments.
Applicant's arguments regarding Bhoyar on page 11 of the Remarks filed 06/29/2026 have been fully considered but they are not persuasive. Applicant argues that Bhoyar’s RGB histogram table is not a 3D color map because Applicant’s recited 3D color map is actively used as a reference space to identify visually corresponding colors around a selected/reference pixel. As stated in independent claims 1, 22, 31, and 33: “the 3D colour referential map is a preselected 3D colour scheme”, and further specified in claims 34-37: “wherein the 3D colour scheme is RGB (Red, Green, Blue), HLS (Hue, Lightness, Saturation) or HVS (Hue, Value, Saturation)”. Thus, Bhoyar teaches the 3D referential colour map ([2.3 Computing JND Histogram] computing histogram of a colour image in RGB space) and identifying in the 3D colour referential map a cluster of the pixels or voxels with similar visual appearance by pixel or voxel value ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Applicant further argues on page 11 that the Office Action conflates agglomeration with highlighting/ obscuring by altering a colour scheme. The amended claim language is: vi) within the image or a part thereof, modifying each said pixel or voxel in the cluster of pixels or voxels to be of said assigned colour if each pixel or voxel in the cluster of pixels or voxels is not of said assigned colour, thereby highlighting (or obscuring or masking, respectively) the pixels or voxels in the cluster. Bhoyar’s agglomeration technique reduces the number of colours by combining the smaller segments with similar coloured larger segments, modifying the colors of the pixels in the smaller segments. As shown in Fig. 1-6 of Bhoyar, agglomeration between adjacent pixels can make details more or less visible locally, thus highlighting or obscuring. Applicant’s further arguments in view of the amendments to independent claims 1, 22, 31, and 33 regarding amended steps iv) and v) been considered but are moot because the new ground of rejection does not rely on the references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument, as facilitated by the newly added amendments. Please see the updated 35 U.S.C. § 103 rejections of independent claims 1, 22, 31, and 33 in view of the newly added amendments.
Applicant's arguments regarding Radha on pages 12-13 of the Remarks filed 06/29/2026 have been fully considered but they are not persuasive. Applicant specifically argues that Radha does not assign colors based on Ip(x,y) and does not make assignments based on an order of each of the colours in a selected sequence of colors.
As stated on pages 14-15 of the Non-Final Rejection filed 03/31/2026:
PNG
media_image1.png
276
676
media_image1.png
Greyscale
PNG
media_image2.png
414
650
media_image2.png
Greyscale
The cited portion of Radha describes locally adjusting picture elements using Ip(x,y), where Ip(x,y) specifically is the color level for the output pixel. The local contrast stretching is performed by sliding a kernel across the image, thus following an order. Therefore, Radha teaches the claim limitation of “assigning colours to at least some pixels or voxels in the image or part thereof based on the relative intensity values and an order of each of the colours in the sequence.”
Applicant’s arguments filed 06/29/2026 with respect to Jin in claims 4 and 25 have been fully considered and are persuasive. The 35 U.S.C. § 103 rejections of claims 4 and 25 have been withdrawn. Claims 5 and 7 depend from claim 4 and claims 27-29 depend from claim 25; therefore the 35 U.S.C. § 103 rejections of claims 5, 7, 27-29 are also withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 8-9, 26, 31-34, and 36-37 are rejected under 35 U.S.C. 103 as being unpatentable over Bhoyar (Bhoyar, Kishor, and Omprakash Kakde. "Colour image segmentation using fast fuzzy c-means algorithm." ELCVIA: electronic letters on computer vision and image analysis (2010): 18-31) in view of Fukazawa (US20130063467A1).
Regarding claim 1, Bhoyar teaches a computer-implemented image processing method comprising ([Abstract] This paper proposes modified FCM (Fuzzy C-Means) approach to colour image segmentation using JND (Just Noticeable Difference) histogram):
i) receiving an image comprising pixels or voxels; ii) identifying the pixels or voxels of the image in a 3D colour referential map (RGB), wherein the 3D colour referential map is a preselected 3D colour scheme ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1. The number of rows in Table 1 is equal to the number of different colour shades available in the image. In Table 1 there will be one entry for each colour shade while in Table 2 there will be one entry for each pixel);
iii) identifying in the 3D colour referential map a cluster of the pixels or voxels with similar visual appearance by pixel or voxel value ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii)Starting from the first colour in Table 1, compare the colour with the next colour in table 1);
iv) retrieving a selection of colours of the pixels or voxels in the cluster ([3 Histogram Agglomeration] ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1. iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger);
v) assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels ([3 Histogram Agglomeration] iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger)
and repeating steps iii) to v) ([3 Histogram Agglomeration] vi) Step ii, iii and iv are repeated for every colour in the Table 1);
vi) within the image or a part thereof, modifying each said pixel or voxel in the cluster of pixels or voxels to be of said assigned colour if each pixel or voxel in the cluster of pixels or voxels is not of said assigned colour, thereby highlighting the pixels or voxels in the cluster ([3 Histogram Agglomeration] After the compressed histogram of a real life image is obtained using the basic JND histogram algorithm given in section 2, the agglomeration or region merging technique can further be used to reduce the number of colours by combining the smaller segments (less than 0.1% of the image size) with similar coloured larger segments…vii) Steps ii to v are repeated till the Table 1 does not reduce further i.e. equilibrium has reached), thereby highlighting the pixels or voxels in the cluster (see Fig. 1-6, agglomeration between adjacent pixels can make details more or less visible locally);
vii) outputting the image as a modified image that includes the modified pixels or voxels (see Fig. 1-6);
and wherein the minimum perceptible pixel or voxel value difference is a colour difference between a first pixel or voxel and a second pixel or voxel that is furthest away in colour from the first pixel or voxel beyond which the user can clearly perceive a difference in colour from the first pixel ([Algorithm 1: JND histogram computing algorithm] i) Select a proper similarity threshold (histogram binning threshold), Θ1 (JNDeye2 ≤ Θ1≤ JNDh2), depending on the precision of vision from fine to broad as required by the application. [2.2 Approximating the value of colour similarity threshold (JND)] Let C1 and C2 be two RGB colours in the new quantized space. Let C1= (Jr1,Jg1,Jb1) =(0,0,0) and its immediate JND neighbour, that is 1 noticeable difference away is C2= (Jr2,Jg2,Jb2) =(255/24,255/28,255/26). Hence JNDeye= sqrt ((255/24)^2 + (255/28)^2 + (255/26)^2)) = sqrt (285.27). Using equation (1) the squared JND threshold of human perception is given by equation 2: Θ = JNDh2 = 2567).
Bhoyar does not explicitly teach iv) retrieving a selection of colours of the pixels or voxels in the cluster and presenting the selection of colours to a user; v) responding to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels, or responding to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input.
Fukazawa, in the same field of endeavor of assigning colors to an image, teaches iv) retrieving a selection of colours of the pixels or voxels in the cluster and presenting the selection of colours to a user ([0036] When a user selects a prescribed representative color, from among the N selection candidates displayed in the selection candidate display region CLT, a plurality of colors included in a sample group to which this representative color belongs is displayed. Additionally from among this plurality of colors, the user can specify the colors used in an image process such as color correction. A plurality of colors selectable as the colors to be processed, that is, a plurality of selection candidates, is clustered beforehand into N sample groups. Accordingly, each of the N sample groups is hereinafter called a cluster. [0037] In this way, instead of a plurality of selection candidates all being displayed, this plurality of selection candidates are classified into N clusters, and only N representative colors are displayed. This is equivalent to reducing the number of selection candidates being displayed to N. Therefore, the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed);
v) responding to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels, or responding to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input ([0062] In step S11, the representative color acquisition section 24 selects a prescribed representative color based on an operation of the user. Here, it will be assumed that the representative color CLB, for example, is selected based on an operation of the user. [0092] As shown in FIG. 8, an image, which has color reduced the image of the color reduction processing region F centered at a position indicated by the cursor PM, by using the representative colors of the selection candidates, is displayed in the selection candidate display region CLT. That is, an image, in which the colors of each pixel of the color reduction processing region F have been replaced by the representative colors of the clusters to which these colors belong, is displayed. In this case, the selection candidate display control section 25 may display the selection candidate display region CLT in an enlarged manner, that is, display the color reduced image in an enlarged manner).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Bhoyar with the teachings of Fukazawa to present the selection of colours to a user and respond to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster because "When it is assumed that a user intends to modify the colors of a prescribed part of a displayed image…the color reduction process is a process which selects, from among a plurality of colors included in an image, only N representative colors (hereinafter called the representative colors) as selection candidates of the colors to be processed" [0034] and "the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed, and thereby the difference of colors among the selection candidates can be easily understood" [0037].
Regarding claim 8, Bhoyar and Fukazawa teach the method of claim 1. Bhoyar further teaches automatically identifying another cluster in the 3D colour referential map of pixels or voxels with similar visual appearance by pixel or voxel value within the image or part thereof ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1. iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger. iv) The merged entry will be removed from Table 1. This reduces number of rows in Table 1. In Table 2 the colour index to be merged is changed by the index to which it is merged. v) Thus the first colour in the Table 1 will be compared with every remaining colour in Table 1 and step ii is repeated if required. vi) Step ii, iii and iv are repeated for every colour in the Table 1. vii) Steps ii to v are repeated till the Table 1 does not reduce further i.e. equilibrium has reached. viii) Table 2 is sorted in ascending order of the colour index).
Regarding claim 9, Bhoyar and Fukazawa teach the method of claim 1. Bhoyar further teaches assigning different colours to the identified cluster of pixels or voxels ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1. iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger. iv) The merged entry will be removed from Table 1. This reduces number of rows in Table 1. In Table 2 the colour index to be merged is changed by the index to which it is merged. v) Thus the first colour in the Table 1 will be compared with every remaining colour in Table 1 and step ii is repeated if required. vi) Step ii, iii and iv are repeated for every colour in the Table 1. vii) Steps ii to v are repeated till the Table 1 does not reduce further i.e. equilibrium has reached. viii) Table 2 is sorted in ascending order of the colour index. [3 Histogram Agglomeration] After the compressed histogram of a real life image is obtained using the basic JND histogram algorithm given in section 2, the agglomeration or region merging technique can further be used to reduce the number of colours by combining the smaller segments (less than 0.1% of the image size) with similar coloured larger segments).
Regarding claim 26, Bhoyar and Fukazawa teach the method of claim 1. Bhoyar further teaches a non-transitory computer readable medium storing a computer program that comprises instructions that, when executed on a computing device, control the device to perform the method (Table 3: Average Performance on BSD (*On AMD Athlon 1.61 GHz processor, 1GB RAM and MATLAB 7 running on Windows XP)).
Regarding claim 31, Bhoyar teaches a computer-implemented image processing method comprising ([Abstract] This paper proposes modified FCM (Fuzzy C-Means) approach to colour image segmentation using JND (Just Noticeable Difference) histogram):
i) receiving an image comprising pixels or voxels; ii) identifying the pixels or voxels of the image in a 3D colour referential map (RGB), wherein the 3D colour referential map is a preselected 3D colour scheme ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1. The number of rows in Table 1 is equal to the number of different colour shades available in the image. In Table 1 there will be one entry for each colour shade while in Table 2 there will be one entry for each pixel);
iii) identifying in the 3D colour referential map a cluster of the pixels or voxels with similar visual appearance by pixel or voxel value ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii)Starting from the first colour in Table 1, compare the colour with the next colour in table 1);
iv) retrieving a selection of colours of the pixels or voxels in the cluster ([3 Histogram Agglomeration] ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1. iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger);
v) assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels ([3 Histogram Agglomeration] iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger)
and repeating steps iii) to v) ([3 Histogram Agglomeration] vi) Step ii, iii and iv are repeated for every colour in the Table 1);
vi) within the image or a part thereof, modifying each said pixel or voxel in the cluster of pixels or voxels to be of said assigned colour if each pixel or voxel in the cluster of pixels or voxels is not of said assigned colour, thereby obscuring or masking the pixels or voxels in the cluster ([3 Histogram Agglomeration] After the compressed histogram of a real life image is obtained using the basic JND histogram algorithm given in section 2, the agglomeration or region merging technique can further be used to reduce the number of colours by combining the smaller segments (less than 0.1% of the image size) with similar coloured larger segments…vii) Steps ii to v are repeated till the Table 1 does not reduce further i.e. equilibrium has reached), thereby highlighting the pixels or voxels in the cluster (see Fig. 1-6, agglomeration between adjacent pixels can make details more or less visible locally);
and vii) outputting the image as a modified image that includes the modified pixels or voxels (see Fig. 1-6);
wherein the minimum perceptible pixel or voxel value difference is a colour difference between a first pixel or voxel and a second pixel or voxel that is furthest away in colour from the first pixel or voxel beyond which the user can clearly perceive a difference in colour from the first pixel ([Algorithm 1: JND histogram computing algorithm] i) Select a proper similarity threshold (histogram binning threshold), Θ1 (JNDeye2 ≤ Θ1≤ JNDh2), depending on the precision of vision from fine to broad as required by the application. [2.2 Approximating the value of colour similarity threshold (JND)] Let C1 and C2 be two RGB colours in the new quantized space. Let C1= (Jr1,Jg1,Jb1) =(0,0,0) and its immediate JND neighbour, that is 1 noticeable difference away is C2= (Jr2,Jg2,Jb2) =(255/24,255/28,255/26). Hence JNDeye= sqrt ((255/24)^2 + (255/28)^2 + (255/26)^2)) = sqrt (285.27). Using equation (1) the squared JND threshold of human perception is given by equation 2: Θ = JNDh2 = 2567).
Bhoyar does not explicitly teach iv) retrieving a selection of colours of the pixels or voxels in the cluster and presenting the selection of colours to a user; v) responding to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels, or responding to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input.
Fukazawa, in the same field of endeavor of assigning colors to an image, teaches iv) retrieving a selection of colours of the pixels or voxels in the cluster and presenting the selection of colours to a user ([0036] When a user selects a prescribed representative color, from among the N selection candidates displayed in the selection candidate display region CLT, a plurality of colors included in a sample group to which this representative color belongs is displayed. Additionally from among this plurality of colors, the user can specify the colors used in an image process such as color correction. A plurality of colors selectable as the colors to be processed, that is, a plurality of selection candidates, is clustered beforehand into N sample groups. Accordingly, each of the N sample groups is hereinafter called a cluster. [0037] In this way, instead of a plurality of selection candidates all being displayed, this plurality of selection candidates are classified into N clusters, and only N representative colors are displayed. This is equivalent to reducing the number of selection candidates being displayed to N. Therefore, the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed);
v) responding to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels, or responding to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input ([0062] In step S11, the representative color acquisition section 24 selects a prescribed representative color based on an operation of the user. Here, it will be assumed that the representative color CLB, for example, is selected based on an operation of the user. [0092] As shown in FIG. 8, an image, which has color reduced the image of the color reduction processing region F centered at a position indicated by the cursor PM, by using the representative colors of the selection candidates, is displayed in the selection candidate display region CLT. That is, an image, in which the colors of each pixel of the color reduction processing region F have been replaced by the representative colors of the clusters to which these colors belong, is displayed. In this case, the selection candidate display control section 25 may display the selection candidate display region CLT in an enlarged manner, that is, display the color reduced image in an enlarged manner).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Bhoyar with the teachings of Fukazawa to present the selection of colours to a user and respond to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster because "When it is assumed that a user intends to modify the colors of a prescribed part of a displayed image…the color reduction process is a process which selects, from among a plurality of colors included in an image, only N representative colors (hereinafter called the representative colors) as selection candidates of the colors to be processed" [0034] and "the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed, and thereby the difference of colors among the selection candidates can be easily understood" [0037].
Regarding claim 32, Bhoyar and Fukazawa teach the method of claim 31. Bhoyar further teaches a non-transitory computer readable medium storing a computer program that comprises instructions that, when executed on a computing device, control the device to perform the method (Table 3: Average Performance on BSD (*On AMD Athlon 1.61 GHz processor, 1GB RAM and MATLAB 7 running on Windows XP)).
Regarding claim 33, Bhoyar teaches an image processing system, the system comprising a processor configured to (Table 3: Average Performance on BSD (*On AMD Athlon 1.61 GHz processor, 1GB RAM and MATLAB 7 running on Windows XP)):
i) receive an image comprising pixels or voxels; ii) identify the pixels or voxels of the image in a 3D colour referential map (RGB), wherein the 3D colour referential map is a preselected 3D colour scheme ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1. The number of rows in Table 1 is equal to the number of different colour shades available in the image. In Table 1 there will be one entry for each colour shade while in Table 2 there will be one entry for each pixel);
iii) identify in the 3D colour referential map a cluster of the pixels or voxels with similar visual appearance by pixel or voxel value ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii)Starting from the first colour in Table 1, compare the colour with the next colour in table 1)
iv) retrieve a selection of colours of the pixels or voxels in the cluster ([3 Histogram Agglomeration] ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1. iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger);
v) assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels ([3 Histogram Agglomeration] iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger)
and repeating steps iii) to v) ([3 Histogram Agglomeration] vi) Step ii, iii and iv are repeated for every colour in the Table 1);
vi) within the image or a part thereof, modify each said pixel or voxel in the cluster of pixels or voxels to be of said assigned colour if each pixel or voxel in the cluster of pixels or voxels is not of said assigned colour, thereby obscuring or masking the pixels or voxels in the cluster ([3 Histogram Agglomeration] After the compressed histogram of a real life image is obtained using the basic JND histogram algorithm given in section 2, the agglomeration or region merging technique can further be used to reduce the number of colours by combining the smaller segments (less than 0.1% of the image size) with similar coloured larger segments…vii) Steps ii to v are repeated till the Table 1 does not reduce further i.e. equilibrium has reached), thereby highlighting the pixels or voxels in the cluster (see Fig. 1-6, agglomeration between adjacent pixels can make details more or less visible locally);
and vii) output the image as a modified image that includes the modified pixels or voxels (see Fig. 1-6);
and wherein the minimum perceptible pixel or voxel value difference is a colour difference between a first pixel or voxel and a second pixel or voxel that is furthest away in colour from the first pixel or voxel beyond which the user can clearly perceive a difference in colour from the first pixel ([Algorithm 1: JND histogram computing algorithm] i) Select a proper similarity threshold (histogram binning threshold), Θ1 (JNDeye2 ≤ Θ1≤ JNDh2), depending on the precision of vision from fine to broad as required by the application. [2.2 Approximating the value of colour similarity threshold (JND)] Let C1 and C2 be two RGB colours in the new quantized space. Let C1= (Jr1,Jg1,Jb1) =(0,0,0) and its immediate JND neighbour, that is 1 noticeable difference away is C2= (Jr2,Jg2,Jb2) =(255/24,255/28,255/26). Hence JNDeye= sqrt ((255/24)^2 + (255/28)^2 + (255/26)^2)) = sqrt (285.27). Using equation (1) the squared JND threshold of human perception is given by equation 2: Θ = JNDh2 = 2567).
Bhoyar does not explicitly teach iv) retrieve a selection of colours of the pixels or voxels in the cluster and present the selection of colours to a user; v) respond to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels, or respond to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input.
Fukazawa, in the same field of endeavor of assigning colors to an image, teaches iv) retrieve a selection of colours of the pixels or voxels in the cluster and present the selection of colours to a user ([0036] When a user selects a prescribed representative color, from among the N selection candidates displayed in the selection candidate display region CLT, a plurality of colors included in a sample group to which this representative color belongs is displayed. Additionally from among this plurality of colors, the user can specify the colors used in an image process such as color correction. A plurality of colors selectable as the colors to be processed, that is, a plurality of selection candidates, is clustered beforehand into N sample groups. Accordingly, each of the N sample groups is hereinafter called a cluster. [0037] In this way, instead of a plurality of selection candidates all being displayed, this plurality of selection candidates are classified into N clusters, and only N representative colors are displayed. This is equivalent to reducing the number of selection candidates being displayed to N. Therefore, the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed);
v) respond to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels, or respond to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input ([0062] In step S11, the representative color acquisition section 24 selects a prescribed representative color based on an operation of the user. Here, it will be assumed that the representative color CLB, for example, is selected based on an operation of the user. [0092] As shown in FIG. 8, an image, which has color reduced the image of the color reduction processing region F centered at a position indicated by the cursor PM, by using the representative colors of the selection candidates, is displayed in the selection candidate display region CLT. That is, an image, in which the colors of each pixel of the color reduction processing region F have been replaced by the representative colors of the clusters to which these colors belong, is displayed. In this case, the selection candidate display control section 25 may display the selection candidate display region CLT in an enlarged manner, that is, display the color reduced image in an enlarged manner).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the system of Bhoyar with the teachings of Fukazawa to present the selection of colours to a user and respond to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster because "When it is assumed that a user intends to modify the colors of a prescribed part of a displayed image…the color reduction process is a process which selects, from among a plurality of colors included in an image, only N representative colors (hereinafter called the representative colors) as selection candidates of the colors to be processed" [0034] and "the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed, and thereby the difference of colors among the selection candidates can be easily understood" [0037].
Regarding claim 34, Bhoyar and Fukazawa teach the method of claim 1. Bhoyar further teaches wherein the 3D colour scheme is RGB (Red, Green, Blue), HLS (Hue, Lightness, Saturation) or HVS (Hue, Value, Saturation) ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1).
Regarding claim 36, Bhoyar and Fukazawa teach the method of claim 31. Bhoyar further teaches wherein the 3D colour scheme is RGB (Red, Green, Blue), HLS (Hue, Lightness, Saturation) or HVS (Hue, Value, Saturation) ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1).
Regarding claim 37, Bhoyar and Fukazawa teach the system of claim 33. Bhoyar further teaches wherein the 3D colour scheme is RGB (Red, Green, Blue), HLS (Hue, Lightness, Saturation) or HVS (Hue, Value, Saturation) ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1).
Claims 3 and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Bhoyar in view of Fukazawa and Radha (Radha, N., and M. Tech. "Comparison of contrast stretching methods of image enhancement techniques for acute leukemia images." Int. J. Eng. Res. Technol 1.6 (2012): 1-9).
Regarding claim 3, Bhoyar and Fukazawa teach the method of claim 1. Radha, in the same field of endeavor of pixel color adjustment, teaches wherein the image is a medical image ([Abstract] medical professional using medical images to diagnose leukemia).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Bhoyar with the teachings of Radha to use medical images because "there are blurness and effects of unwanted noise on blood leukaemia images that sometimes result in false diagnosis. Thus image pre-processing such as image enhancement techniques are needed to improve this situation. This project proposes several contrast enhancement techniques" [Radha Abstract].
Regarding claim 10, Bhoyar and Fukazawa teach the method of claim 9. Radha, in the same field of endeavor of pixel color adjustment, teaches wherein assigning colours to the image or part thereof, comprises: selecting a sequence of colours for assignment to the image or part thereof; determining a minimum intensity IMIN within the image or part thereof; determining a maximum intensity IMAX within the image or part thereof; and determining relative intensity values RIV(i) for each pixel or voxel i according to:
R
I
V
i
=
f
I
i
-
I
M
I
N
I
M
A
X
-
I
M
I
N
where I(i) is an intensity of pixel or voxel i, and f is a preselected function; and assigning colours to at least some pixels or voxels in the image or part thereof based on the relative intensity values and an order of each of the colours in the sequence.
PNG
media_image3.png
417
466
media_image3.png
Greyscale
PNG
media_image4.png
450
477
media_image4.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Bhoyar with the teachings of Radha to assign colors to pixels based on the calculated relative intensity because "Improvement in quality of medical images can be achieved by using Contrast stretching" [Radha pg. 2].
Regarding claim 11, Bhoyar, Fukazawa, and Radha teach the method of claim 10. Radha further teaches comprising: (i) selecting intensity values in the original image and applying the method only to pixels or voxels of the intensity values thus selected; (ii) selecting automatically the image or a part thereof based on one or more criteria; and/or (iii) generating a new image by colouring the image or part thereof according to the sequence of colours ([2.2.1 Local and global contrast stretching] Local contrast stretching (LCS) is an enhancement method performed on an image for locally adjusting each picture element value to improve the visualization of structures in both darkest and lightest portions of the image at the same time. LCS is performed by sliding windows (called the KERNEL) across the image and adjusting the center element. [3.1. Results for local contrast stretching] Figure 4, Figure 5, Figure 6 shows original the three images. Meanwhile, the results for each normal, bright and dark image for Local Contrast Stretching technique are shown in, Figure 7, Figure 8 and Figure 9).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Bhoyar with the teachings of Radha to generate a new image by coloring according to a sequence of colors because "The resultant, images become clearer and the features of leukemia cells can easily been seen and improved from the original for each category. Nucleus and cytoplasm of immature white blood cells become clearer. Hence, they can easily been discussed by hematologists" [Radha pg. 4].
Claims 22, 30, and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Bhoyar in view of Fukazawa and Mitsuzaki (JP2011099971A).
Regarding claim 22, Bhoyar teaches an image processing system, the system comprising a processor configured to (Table 3: Average Performance on BSD (*On AMD Athlon 1.61 GHz processor, 1GB RAM and MATLAB 7 running on Windows XP)):
i) receive an image comprising pixels or voxels; ii) identify the pixels or voxels of the image in a 3D colour referential map (RGB), wherein the 3D colour referential map is a preselected 3D colour scheme ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1. The number of rows in Table 1 is equal to the number of different colour shades available in the image. In Table 1 there will be one entry for each colour shade while in Table 2 there will be one entry for each pixel);
iii) identify in the 3D colour referential map a cluster of the pixels or voxels with similar visual appearance by pixel or voxel value ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1);
iv) retrieve a selection of colours of the pixels or voxels in the cluster ([3 Histogram Agglomeration] ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1. iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger);
v) assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels ([3 Histogram Agglomeration] iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger)
and repeating steps iii) to v) ([3 Histogram Agglomeration] vi) Step ii, iii and iv are repeated for every colour in the Table 1);
vi) within the image or a part thereof, modify each said pixel or voxel in the cluster of pixels or voxels to be of said assigned colour if each said pixel or voxel in the cluster of pixels or voxels is not of said assigned colour ([3 Histogram Agglomeration] After the compressed histogram of a real life image is obtained using the basic JND histogram algorithm given in section 2, the agglomeration or region merging technique can further be used to reduce the number of colours by combining the smaller segments (less than 0.1% of the image size) with similar coloured larger segments), thereby highlighting the pixels or voxels in the cluster (see Fig. 1-6, agglomeration between adjacent pixels can make details more or less visible locally);
vii) output the image as a modified image that includes the modified pixels or voxels (see Fig. 1-6);
and wherein the minimum perceptible pixel or voxel value difference is a colour difference between a first pixel or voxel and a second pixel or voxel that is furthest away in colour from the first pixel or voxel beyond which the user can clearly perceive a difference in colour from the first pixel ([Algorithm 1: JND histogram computing algorithm] i) Select a proper similarity threshold (histogram binning threshold), Θ1 (JNDeye2 ≤ Θ1≤ JNDh2), depending on the precision of vision from fine to broad as required by the application. [2.2 Approximating the value of colour similarity threshold (JND)] Let C1 and C2 be two RGB colours in the new quantized space. Let C1= (Jr1,Jg1,Jb1) =(0,0,0) and its immediate JND neighbour, that is 1 noticeable difference away is C2= (Jr2,Jg2,Jb2) =(255/24,255/28,255/26). Hence JNDeye= sqrt ((255/24)^2 + (255/28)^2 + (255/26)^2)) = sqrt (285.27). Using equation (1) the squared JND threshold of human perception is given by equation 2: Θ = JNDh2 = 2567).
Bhoyar does not explicitly teach iv) retrieve a selection of colours of the pixels or voxels in the cluster and presenting the selection of colours to a user; v) respond to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels, and respond to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input.
Fukazawa, in the same field of endeavor of assigning colors to an image, teaches iv) retrieving a selection of colours of the pixels or voxels in the cluster and presenting the selection of colours to a user ([0036] When a user selects a prescribed representative color, from among the N selection candidates displayed in the selection candidate display region CLT, a plurality of colors included in a sample group to which this representative color belongs is displayed. Additionally from among this plurality of colors, the user can specify the colors used in an image process such as color correction. A plurality of colors selectable as the colors to be processed, that is, a plurality of selection candidates, is clustered beforehand into N sample groups. Accordingly, each of the N sample groups is hereinafter called a cluster. [0037] In this way, instead of a plurality of selection candidates all being displayed, this plurality of selection candidates are classified into N clusters, and only N representative colors are displayed. This is equivalent to reducing the number of selection candidates being displayed to N. Therefore, the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed);
v) responding to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster of pixels or voxels ([0062] In step S11, the representative color acquisition section 24 selects a prescribed representative color based on an operation of the user. Here, it will be assumed that the representative color CLB, for example, is selected based on an operation of the user. [0092] As shown in FIG. 8, an image, which has color reduced the image of the color reduction processing region F centered at a position indicated by the cursor PM, by using the representative colors of the selection candidates, is displayed in the selection candidate display region CLT. That is, an image, in which the colors of each pixel of the color reduction processing region F have been replaced by the representative colors of the clusters to which these colors belong, is displayed. In this case, the selection candidate display control section 25 may display the selection candidate display region CLT in an enlarged manner, that is, display the color reduced image in an enlarged manner).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the system of Bhoyar with the teachings of Fukazawa to present the selection of colours to a user and respond to a user input that controls confirmation of the selection of colours by assigning a colour from the selection of colours to each pixel or voxel in the cluster because "When it is assumed that a user intends to modify the colors of a prescribed part of a displayed image…the color reduction process is a process which selects, from among a plurality of colors included in an image, only N representative colors (hereinafter called the representative colors) as selection candidates of the colors to be processed" [0034] and "the user can select, by fewer work operations, a prescribed color from among all the colors constituting an object to be processed, and thereby the difference of colors among the selection candidates can be easily understood" [0037]
Mitsuzaki, in the same field of endeavor of image color analysis teaches respond to a user input that controls adjustment of the selection of colours by adjusting the minimum perceptible pixel or voxel value difference according to the user input ([pg. 4-5] The image quality adjustment designation unit 213 inputs an instruction from the user for image quality setting and image adjustment (as will be described in detail later, this image quality adjustment designation unit 213 is the first and second image setting control unit in this example). For example, the user can input the following through the image quality adjustment designation unit 213…Color matching: “Bright”, “Photo”, “Minimum color difference” type settings • Color balance: Independent adjustment of each color in several steps. [pg. 5 para. 8] When an attribute determination signal for each pixel is developed and input, a color matching type table optimized for each target object type is applied while the attribute determination unit 221 sequentially determines the attribute).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the system of Bhoyar with the teachings of Mitsuzaki to respond to a user input to control the adjustment of the selection of colors by adjusting the minimum perceptible pixel or voxel value difference according to the user input so that "In this way, the image quality setting mode in which data processing such as color correction and screen processing is performed on the image data, and (2) items that are adjusted by the user, such as density, color balance, and contrast, are appropriately printed according to the adjustment contents" [pg. 2 para. 3].
Regarding claim 30, Bhoyar, Fukazawa, and Mitsuzaki teach the system of claim 22. Bhoyar further teaches wherein the processor is further configured to assign different colours to the identified cluster of pixels or voxels ([3 Histogram Agglomeration] Agglomeration in chemical processes attributes to formation of bigger lumps from smaller particles. In the digital image segmentation, the similar pixels (in some sense) are clustered together under some similarity criteria…ii) Starting from the first colour in Table 1, compare the colour with the next colour in table 1. iii) If the population of the smaller segment is smaller than .1% and the two segments are similar using θ2 , merge the ith colour with the previous one (the first in Table 1), their populations will be added and the colour of larger population will represent the merger. iv) The merged entry will be removed from Table 1. This reduces number of rows in Table 1. In Table 2 the colour index to be merged is changed by the index to which it is merged. v) Thus the first colour in the Table 1 will be compared with every remaining colour in Table 1 and step ii is repeated if required. vi) Step ii, iii and iv are repeated for every colour in the Table 1. vii) Steps ii to v are repeated till the Table 1 does not reduce further i.e. equilibrium has reached. viii) Table 2 is sorted in ascending order of the colour index. [3 Histogram Agglomeration] After the compressed histogram of a real life image is obtained using the basic JND histogram algorithm given in section 2, the agglomeration or region merging technique can further be used to reduce the number of colours by combining the smaller segments (less than 0.1% of the image size) with similar coloured larger segments).
Regarding claim 35, Bhoyar, Fukazawa, and Mitsuzaki teach the system of claim 22. Bhoyar further teaches wherein the 3D colour scheme is RGB (Red, Green, Blue), HLS (Hue, Lightness, Saturation) or HVS (Hue, Value, Saturation) ([2.3 Computing JND Histogram] In this section, we propose an algorithm for computing histogram of a colour image in RGB space…Thus Table 1 contains the R, G, B coordinates and the respective frequency information or population (H) of the tri-colour stimulus, while Table 2 contains the x and y positional co-ordinates in the image and respective colour index (row index) in Table 1).
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Bhoyar in view of Fukazawa, Mitsuzaki and Radha.
Regarding claim 24, Bhoyar, Fukazawa, and Mitsuzaki teach the system of claim 22. Radha, in the same field of endeavor of pixel color adjustment, teaches wherein the image is a medical image ([Abstract] medical professional using medical images to diagnose leukemia).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the system of Bhoyar with the teachings of Radha to use medical images because "there are blurness and effects of unwanted noise on blood leukaemia images that sometimes result in false diagnosis. Thus image pre-processing such as image enhancement techniques are needed to improve this situation. This project proposes several contrast enhancement techniques" [Radha Abstract].
Allowable Subject Matter
Claims 4, 5, 7, 25, and 27-29 are objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and if all withstanding rejections and objections were overcome. Regarding claims 4 and 25, the following limitations were not found to be taught in the art: wherein identifying the cluster of pixels or voxels comprises: selecting at least one pixel or voxel value within the image or part thereof; creating a pixel or voxel value profile curve from a reference point (xi, 0) corresponding to the at least one pixel or voxel value to all pixel or voxel values in a selected pixel or voxel value scheme; detecting a closest point (s1) on the pixel or voxel value profile curve to the reference point (xi, 0); locating a point of inflexion on the pixel or voxel value profile curve between the reference point (xi, 0) and the closest point (s1); and segmenting a portion of the pixel or voxel value profile curve between the reference point (xi, 0) and the point of inflexion, the segmented portion of the pixel or voxel value profile curve constituting a cluster of pixel or voxel values corresponding to the at least one pixel or voxel value. Claims 5 and 7 depend from claim 4 and claims 27-29 depend from claim 25 and therefore would also be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and if all withstanding rejections and objections were overcome.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jacqueline R Zak whose telephone number is (571)272-4077. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACQUELINE R ZAK/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights