DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
2. This communication is in response to the application filled on 07/13/2023.
3. Claims 1-3, 5-20 are pending.
4. Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Information Disclosure Statement
5. The information disclosure statements (IDS) submitted on 07/13/2023 and 07/16/2024 have been considered by the examiner.
Continued Examination Under 37 CFR 1.114
6. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/30/2026 has been entered.
Response to Arguments
7. Applicant’s arguments, see pg. 1-3, with regard to the 102 rejection of claim 1-2, 4-7, and 10-12 have been fully considered and persuasive. Specifically, the examiner notes Kimura fails to specifically disclose wherein the pieces of held light source spectral information correspond to previously measured light sources. Therefore, the 102 rejections of claims claim 1-2, 4-7, and 10-12 are withdrawn in view of Kimura failing to specifically disclose held light source spectral information corresponding to previously measured light sources. However, upon further consideration, a new ground of rejection is made in view of JP-2017215851-A to Kimura and further in view of U.S. Publication No. 2021/0274102 to Kadoi et al.
Claim Rejections - 35 USC § 103
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1-2, 5-7, 10-13, 15-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable by JP-2017215851-A to Kimura (hereinafter Kimura) and further in view of U.S. Publication No. 2021/0274102 to Kadoi et al.
10. Regarding Claim 1, Kimura discloses an image processing device comprising ([Fig. 2(a) and (b)], [par. 0012, ln. 1-4] “The camera system 100 includes… control unit 5, an image pickup device 6, an image processing unit 7, a memory unit 8…”):
a memory storing instructions ([Fig. 2(a) and (b)], [par. 0012, ln. 1-4], [par. 0014, ln. 1-3] “The camera control unit 5 loads the program stored in the nonvolatile portion of the memory to the volatile portion of the memory and executes the program to control the operation of the camera body 1 and the lens unit 2 so that the operation of the entire camera system 100 Is realized.”), and
at least one processor configured to execute the instructions to perform operations comprising ([Fig. 2(a) and (b)], [par. 0012, ln. 1-4], [par. 0014, ln. 1-3]):
detecting a specific region that is a region of an image of a specific object from an image of a subject to which light from a light source is emitted ([par. 0046, ln. 1-2] “In S undefined 150, the image processing unit 7 estimates a corresponding point between the multi-view images. Details will be described later.”, [Fig. 5(b)], [par. 0051, ln. 1-8] “…details of… S150… shown in FIG. 5 (b). The estimation of the corresponding points is a process of searching relative positional relationships of arbitrary two images among the multi-viewpoint images generated in S140. A part of one of the images is cut out into a rectangular (block) shape and used as a template. Then, a search area is set in the other image, and the similarity (correlation) is calculated at each position while moving the template in the search area, and a position with the highest similarity is searched… it is possible to detect corresponding points in each of the other images with respect to each pixel of the reference image by estimating corresponding points with each of the multi-view images as a reference image.”, [par. 0054, ln. 1-10] “…image processing unit 7 searches for an area having the highest degree of similarity with the template 93 in the image 94… 7 searches for an area having the smallest sum of absolute differences (SAD) between… 93 and the pixel value among the areas in… 94 as a region having the highest degree of similarity with… 93… FIG. 6… 7 first calculates the degree of similarity with… 93 for the area 95 in … 94 corresponding to the cutout position of… 93 of the image 91. Thereafter, as indicated by an arrow 96… 7 calculates the similarity at each position while successively moving the position of the area for calculating the similarity in… 94 in the horizontal and vertical directions…”);
detecting light source color information on a basis of a plurality of image signals of the detected specific region ([Fig. 1(a) and (b)], [par. 0046, ln. 2-3] “In S160, the camera control unit 5 estimates the light source color by using the information of the corresponding point estimated in S150. Details will be described later.”, [par. 0059, ln. 1-7] “…details of… S160 will be described… shown in FIG. 5 (c). In S410, based on the information of the corresponding point obtained by the corresponding point estimating process and the pixel value, the camera control unit 5 calculates the value (color) of a plurality of pixels corresponding to the same position (that is, corresponding point) of the same object ) To the chromaticity diagram. Details of the mapping will be described later with reference to FIG. 1. The mapping is performed for at least one corresponding point included in a region used for light source color estimation.”, [par. 0060, ln. 1-5] “In S430… unit 5 determines an estimated color of the light source based on maximum likelihood estimation or the like. When mapping is performed for a plurality of corresponding points as in an example to be described later with reference to FIG. 1(b), the light source colors estimated individually may not coincide in some cases. In such a case, the camera control unit 5 determines the light source color based on the maximum likelihood estimation such as the least squares method.”); and
generating generates light source spectral information, which is information of a spectrum of the light source, on a basis of the detected light source color information ([par. 0032, ln. 1-7] “FIG. 4 (a)… showing the reflected light (diffuse reflected light) 52 of the light beam 51 in the case where the surface of the object 50 is a perfectly diffuse reflection surface (Lambertian surface). In FIG. 4 (a), reflected light 52 is represented by an intensity envelope. The diffuse reflected light on the Lambertian surface has uniform reflection in each direction, and the luminance (intensity) of the reflected light does not depend on the observation direction. The luminance Ld (λ) of diffuse reflected light can be expressed by the following equation. (1) where Ie (λ) is the luminance of the light source, Kd (λ) is the diffuse reflection characteristic, θ is the incident angle, and λ is the wavelength.”, [par. 0047, ln. 2-3] “By estimating the light source color in S160, the light source luminance Ie (λ) in the equations (1) to (3) is found.” The examiner specifically notes that λ is wavelength per [par. 0032, ln. 1-7] and therefore in S160 at least the wavelength of a light source is determined, which is “spectral information”), wherein generating the light source spectral information includes performing selection from a plurality of pieces of held light source spectral information ([par. 0020, ln. 3-7] “…the memory unit 8 stores information on blackbody radiation, for example, information representing a curve indicating blackbody radiation in the xy chromaticity space.”, [par. 0040, ln. 3-5] “However, the light source color estimating operation of the present embodiment can also be executed on image data already recorded in a storage device or the like of the memory unit 8, for example.”, [par. 0064, ln. 1-7] “FIG. 1 (a) is a diagram for explaining a light source color estimation method in a case where it is assumed that the light source can be approximated by blackbody radiation. 71 is a line drawn on the xy chromaticity diagram by the color of the single wavelength light (the color with the highest saturation), 72 to 74 are the coordinates of the color observed when viewing the same point on the object from different viewpoints Chromaticity coordinates), and 75 represents a straight line calculated from the coordinates 72 to 74. In addition, reference numeral 76 denotes a blackbody locus, and reference numeral 77 denotes chromaticity coordinates of the estimated light source color.”, [par. 0065, ln. 1-10] “A black body is an ideal object that completely absorbs and radiates the energy of electromagnetic waves incident from the outside irrespective of the wavelength. If the absolute temperature [K] of the blackbody is determined, the blackbody It is an object whose spectral distribution (ie, color) is uniquely determined. It has a reddish color at a low color temperature (about 1000 K), a white color at an intermediate color temperature (about 6000 K), and a bluish color at a high color temperature (about 10000 K). The blackbody locus 76 shows such a color change of the black body radiation corresponding to the color temperature in the xy chromaticity diagram. It is known that sunlight can be approximated very well by blackbody radiation, and outdoor light sources (except artificial light sources) can be approximated by blackbody radiation. Information representing the blackbody locus on the chromaticity diagram is stored in the memory unit 8…”) {corresponding to previously measured light sources}.
Kimura does not specifically disclose wherein the plurality of pieces of held light source spectral information correspond to previously measured light sources. Specifically, while Kimura discloses selecting form previously held light source spectral information, this selection is based on general light source spectral information (e.g., blackbody radiation, the estimated information of the current light source), which is not directly related to a specific instance of a previously measured light source.
However, Kadoi specifically discloses selection from a plurality of pieces of held light source spectral information corresponding to previously measured light sources ([par. 0048, ln. 1-18] “The color temperature information may be a color temperature representing the color of light emitted by the light source using a corresponding blackbody temperature (unit: kelvin (K)). With respect to a light source whose chromaticity is not present on the blackbody locus, the correlated color temperature may be adopted, and in this case, the color temperature information is information also including a color difference Δuv from the blackbody locus. Also, a chromaticity coordinate value (x, y) in the XYZ color system may be adopted. Also, any form of the color temperature information may be adopted as long as the information is information representing the light source or the color of light emitted by the light source, such as the information obtained by the spectrum of spectral distribution being sampled and recorded. In the following “color temperature” is used as a generic name of information in which a light source or the color of light emitted by the light source is represented using any scale.”, [par. 0049, ln. 1-19] “Note that, if the color temperature of the light source changes due to degradation over time of the light emitting body or the like of the interactive light source 201, the difference between the color temperature in the specification that is transmitted by the interactive light source 201 and the actual color temperature increases. Therefore, information regarding a use period or a use status of the light source may be added to the color temperature map. Also, when it can be determined that the difference between the color temperature in the specification and the actual color temperature is largely based on the use period and the use status of the light source, for example, intrinsic information of the interactive light source 201 may be included in the color temperature map instead of the color temperature. In this case, the color temperature cannot be known from the color temperature map, but it is possible to understand partial areas whose color temperatures are different. Also, luminance information may be added to the color temperature map.”, [par. 0058, ln. 1-12] “The CPU 103 estimates a light source having a color evaluation value whose distance, in the chromaticity diagram, from the color evaluation value obtained by the image processing unit 105 is shortest, out of the color evaluation values similarly obtained from captured images obtained by shooting a white object under a plurality of different types of light sources, in advance. Also, the CPU 103 estimates the color temperature of the specified light source as the color temperature of the non-interactive light source 202 that illuminates the divided area corresponding to the color evaluation value obtained by the image processing unit 105.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Kimura and Kadoi as within the same filed of image processing using spectral characteristics, and as analogous to the claimed invention. Specifically, the motivation to combine would have been obvious to one of ordinary skill in the art, in that by using previously measured light source spectral information, current unknown light source spectral information can be estimated via selection from previously measured light source spectral information ([par. 0054, ln. 3-8] “…However, regarding the non-interactive light source 202, the color temperature needs to be estimated from a captured image. The processing in steps S203 to S205 is processing for estimating the color temperature of the non-interactive light source 202...”, [par. 0058, ln. 1-12]). One of ordinary skill in the art, before the effective filing date of the claimed invention, would have combined the device of Kimura with the selection from previously measured light source spectral information of Kadoi, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 1.
11. Regarding Claim 2, a combination of Kimura and Kadoi teaches the device of claim 1. Kimura further discloses wherein the light source spectral information is information indicating a relationship between a wavelength and the spectrum ([par. 0032, ln. 1-7], [par. 0047, ln. 2-3], [par. 0064, ln. 1-7]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 2.
12. Regarding Claim 5, a combination of Kimura and Kadoi teaches the device of claim 1. Kimura further discloses wherein the operations further comprise: detecting, as the light source color information, an approximate straight line generated on a basis of the plurality of image signals in a chromaticity space ([Fig. 1(a) and (b)], [par. 0064, ln. 1-7], [par. 0070, ln. 1-6] “In FIG. 1 (b)… Coordinates 82 to 84 are colors observed when a point on an object having a color different from the coordinates 72 to 74 is viewed from different viewpoints and 85 is a straight line obtained from coordinates 82 to 84. Coordinates 86 to 88 are colors observed when a point on the object having a color different from the coordinates 72 to 74 and the coordinates 82 to 84 is viewed from different viewpoints and 89 is a line obtained from the coordinates 86 to 88.”, [par. 0071, ln. 1-11] “…FIG. 1 (b)… the point that the light source color is specified as the coordinates on the straight line obtained from a plurality of coordinates whose apparent color differs depending on the magnitude of specular reflection component… FIG. 1 (b), instead of the blackbody locus, a plurality of straight lines relating to different object colors are obtained, and the coordinates of the intersections of the plurality of straight lines are specified as light source colors. Specifically, straight lines 75, 85, and 89 are obtained for a plurality of object colors, and the intersection point of the straight lines is set as the estimated color of the light source. If the straight lines do not intersect at one point, the coordinates 77 which minimizes the sum of the distances from the three straight lines are obtained as the coordinates closest to the intersection point, and it is set as the estimated color of the light source…”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 5.
13. Regarding Claim 6, a combination of Kimura and Kadoi teaches the device of claim 5. Kimura further discloses wherein the operations further comprise: detecting, as the light source color information, a color temperature detected on a basis of the approximate straight line ([Fig. 1(a) and (b)], [par. 0064, ln. 1-7], [par. 0065, ln. 1-10]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 6.
14. Regarding Claim 7, a combination of Kimura and Kadoi teaches the device of claim 6. Kimura further discloses wherein the operations further comprise: detecting, as the color temperature, a color in the chromaticity space at an intersection point of the approximate straight line and a blackbody locus ([Fig. 1(a) and (b)], [par. 0064, ln. 1-7], [par. 0065, ln. 1-10]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 7.
15. Regarding Claim 10, a combination of Kimura and Kadoi teaches the device of claim 1. Kimura further discloses wherein the operations further comprise: adjusting the image of the subject on a basis of the generated light source spectral information ([par. 0023, ln. 1-3] “…as a part of the white balance adjustment processing in the image processing unit 7, it is possible to execute a light source color estimation process to be described later and execute an auto white balance adjustment using a white detection range according to the estimated light source color .”, [par. 0048, ln. 2-6] “The color represented by the component of the diffuse reflection light extracted in this way is influenced by the light source color… by applying the white balance gain based on the light source color estimated in S160, it is possible to estimate the object color by reducing the influence of the light source color from the observed color. For example, when the image pickup device 6 uses a color filter of primary color Bayer array, the signal value corresponding to the transmitted light of the green filter is set as a reference (gain 1.0) and the signal value corresponding to the transmitted light of the red and blue filters”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 10.
16. Regarding Claim 11, the claim language is analogous to claim 1, except “…an image processing method comprising…” wherein the remainder of the claim is analogous to claim 1. Kimura discloses an image processing method ([par. 0001, ln. 1] “The present invention relates to an image processing apparatus and an image processing method”). Rejections analogous to claim 1 are further applicable to the remainder of claim 11. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 11.
17. Regarding Claim 12, the claim language is analogous to claim 1, except “A non-transitory computer readable storage medium storing a program, the program being executable by a processor to perform operations comprising:” wherein the remainder of the claim is analogous to claim 1. Kimura discloses a non-transitory computer readable storage medium storing a program ([Fig. 2(a) and (b)], [par. 0012, ln. 1-4], [par. 0014, ln. 1-4] “The camera control unit 5 loads the program stored in the nonvolatile portion of the memory to the volatile portion of the memory and executes the program to control the operation of the camera body 1 and the lens unit 2 so that the operation of the entire camera system 100 Is realized.”). Rejections analogous to claim 1 are further applicable to the remainder of claim 12. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the non-transitory computer readable medium of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claim 12.
18. Regarding Claims 13, 15-17, and 20, the claim language is analogous to claims 2, 5-7, and 10 respectively. Rejections analogous to claims 2, 5-7, and 10 are further applicable to claims 13, 15-17, and 20 in view of the non-transitory computer readable storage medium of Kimura ([Fig. 2(a) and (b)], [par. 0012, ln. 1-4], [par. 0014, ln. 1-4]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the non-transitory computer readable medium of Kimura with the selection from previously measured light source spectral information of Kadoi to obtain the invention as specified in claims 13, 15-17, and 20.
19. Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over JP-2017215851-A to Kimura and in view of U.S. Publication No. 2021/0274102 to Kadoi, and further in view of U.S. 2006/0139668 to Nishikawa (hereinafter Nishikawa).
20. Regarding Claim 3, a combination of Kimura and Kadoi the device of claim 2. Kimura does not specifically disclose wherein the operations further comprise: generating the light source spectral information by mixing, for each wavelength, spectrums in a plurality of pieces of the light source spectral information based on the detected light source color information.
However, Kadoi teaches to generates the light source spectral information by mixing, {for each wavelength}, spectrums in a plurality of pieces of the light source spectral information based on the detected light source color information ([Fig. 6, see 2002 and 2004], [par. 0090, ln. 1-10] “…when the reflectance of a subject does not largely rely on the color temperature of a light illuminating the subject… color temperature of an overlapped area approaches the color temperature of the light source whose illumination intensity is larger. For example, in an overlapped area illuminated by two light sources of 4000K and 5000K, if the illumination intensity by the light source of 5000K is higher, the color temperature of the overlapped area takes a value that is larger than 4500K (less than 5000K, however).”, [par. 0091, ln. 1-6] “The illumination intensity depends on the distance from the light source, but the distance from the light source to the subject is unknown, and therefore weighted averaging of the color temperatures is performed according to the brightness (luminance) of the subject due to each light source.”, [par. 0092, ln. 1-6, Formula 1] “When two light sources are present, the color temperatures thereof are denoted by K1 and K2, and the luminance thereof are denoted by Y1 and Y2, the combined color temperature Kmix can be obtained using the following Formula 1.
K
m
i
x
=
(
K
1
×
Y
1
+
K
2
×
Y
2
)
/
(
Y
1
+
Y
2
)
”, [par. 0093, ln. 1-9] “When three or more light sources are present, the combined color temperature can be calculated by increasing the number of terms. Note that the unit of the color temperature is not limited to K (Kelvin) as described above. When the temperature in K and a color difference Δuv are set to the map in a combined manner, these values may be applied to Formula 1. Also, Formula 1 may be applied in a uniform perceptual space represented using a unit such as the mired ([M]).”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Kimura and Kadoi as within the same filed of image processing using spectral characteristics, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, in that by incorporating the light source spectral information mixing of Kadoi you effectively eliminate the ambiguity of the light source information for areas illuminated by multiple light sources. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the device of Kimura with the light source mixing and selection from previously measured light source spectral information Kadoi through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Kadoi does not specifically disclose that the mixing is performed “for each wavelength”. Therefore, a combination of Kimura and Kadoi does not specifically disclose that the mixing is performed for each wavelength. However, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that you would perform the mixing “for each wavelength” in combining with Kimura, since the spectral information of Kimura is determined in wavelength units ([par. 0032, ln. 1-7], [par. 0047, ln. 2-3]) and the light sources temperatures are directly proportional and typically calculated from the wavelengths of the respective light sources.
However, Nishikawa specifically teaches wherein spectral characteristics of a color is determined by mixing for each wavelength ([Fig. 9], [par. 0068, ln. 1-14] “Since the spectral reflectance is a reflectance of light by an object for each wavelength, if the reflectance of light of a given object A at an arbitrary wavelength λ is
R
A
(
λ
)
, the energy of reflected light when light of the wavelength λ becomes incident on object A becomes
R
A
(
λ
)
times. Color composition based on the subtract ice process in print processing or the like can be considered a case wherein, for example, light reflected by object A (spectral reflectance
R
A
(
λ
)
) is further reflected by object B (spectral reflectance
R
B
(
λ
)
), and it is considered that the energy of light which becomes
R
A
(
λ
)
times by (the color of) object A further becomes
R
B
(
λ
)
times by (the color) of object B. That is, the spectral reflectance
R
(
λ
)
of the composite color of the two colors is given by
R
A
(
λ
)
×
R
B
(
λ
)
.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Kimura, Kadoi, and Nishikawa as within the same field of image processing using spectral characteristics, and as analogous to the claimed invention. The motivation to combine the device of the combination of Kimura and Kadoi with the for each wavelength mixing of Nishikawa would have been obvious to one of ordinary skill in the art, and is disclosed in Nishikawa ([par. 0005, ln. 1-15] “No scheme for calculating a color obtained by mixing two spot colors (to be referred to as "composite spot color" hereinafter) has been established yet. For example, as a method of reproducing such composite spot color, a method of calculating one color value from the two color values of two spot colors on the DIC space using an arbitrary method may be used. However, this calculation can give an approximately correct value, but it cannot be an accurate calculation method. For example, if a cyan value upon converting a given spot color into a CMYK value is 90%, and that upon converting another spot color into a CMYK value is 80%, a cyan value upon compositing these two spot colors is 170% if their densities are simply added. However, the upper limit of the density value is 100%, and inconsistency occurs.”, [par. 0006, ln. 1-5] “Even in case of the DIC space, since color spaces such as L*a*b*, XYZ, and the like, which are popularly used in color management, have nonlinear characteristics, it is impossible in principle to calculate color composition by means of addition, multiplication, or the like.”), in that performing the mixing in only the color spectrum can result in inaccuracies. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the device of the combination of Kimura and Kadoi with the for each wavelength mixing of Nishikawa through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the effective filling date of the claimed invention, to combine the device of Kimura with the light source mixing and selection from previously measured light source spectral information of Kadoi, and the for each wavelength mixing of Nishikawa to obtain the invention as specified in claim 3.
21. Regarding Claim 14, a combination of Kimura and Kadoi teaches the non-transitory computer readable storage medium of claim 13. The claim language is analogous to claim 3. Rejections analogous to claim 3 are further applicable to claim 14 in view of the non-transitory storage medium of Kimura. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the effective filling date of the claimed invention, to combine the non-transitory computer readable storage medium of Kimura with the light source mixing and selection from previously measured light source spectral information of Kadoi, and the for each wavelength mixing of Nishikawa to obtain the invention as specified in claim 14.
22. Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over JP-2017215851-A to Kimura, in view of U.S. Publication No. 2021/0274102 to Kadoi, and further in view of U.S. Patent No. 9,282,305 to Kikuchi (hereinafter Kikuchi).
23. Regarding Claim 8, a combination of Kimura and Kadoi teaches the device of claim 1. Kimura does not specifically disclose wherein the operations further comprise: detecting the specific region on a basis of a relationship between a wavelength and reflectance of the specific object. Specifically, Kimura detects the region on the basis of template matching as opposed to spectral characteristics of an object. Likewise, Kadoi does not specifically disclose wherein the operations further comprise: detecting the specific region on a basis of a relationship between a wavelength and reflectance of the specific object.
However, Kikuchi teaches wherein a specific region detection unit can detect a specific region on a basis of a relationship between a wavelength and reflectance of the specific object ([col. 10, ln. 54 to col. 11, ln. 38] “…step S405… estimates the spectral reflectivity of the object from a multiband-captured image… The position (x, y) within the captured image, the pixel value g(x, y, b) corresponding to the band b, and the spectral reflectivity t(x, y, λ) of the object have the relationship represented by the following expression (1).
g
x
,
y
,
b
=
∫
f
(
b
,
λ
)
s
λ
e
λ
t
x
,
y
,
λ
d
λ
+
n
(
b
)
(1), where, λ is the wavelength, f(b, λ) is the spectral sensitivity characteristics of the color filter corresponding to the band b, s(λ) is the spectral sensitivity characteristics of the camera, e(λ) is the spectral radiant characteristics of the light source, and n(b) is imaging noise corresponding to the band b… expression (2) obtained by discretizing the expression (1) in the wavelength direction is used for actual calculations.
G
x
,
y
=
F
E
S
T
x
,
y
+
N
(2)… The spectral reflectivity of the object is estimated using the Wiener estimation method… can be calculated using … expression (3)…
T
^
x
,
y
=
R
S
S
F
S
E
t
F
S
E
R
S
S
F
S
E
t
+
R
N
N
-
1
G
x
,
y
”, [col. 14, ln. 20-24] “A modification is described below in which a specific area (object) is detected using the estimated spectral data…”, [col. 14, ln. 55 to col. 15, ln. 7] “The estimated spectrum value data and the detection target spectral data are compared (evaluated) using the difference on a wavelength basis… When comparing spectral data R1 (solid line) and spectral data R2 (broken line) illustrated in FIG. 11, the value E calculated by the following expression (14) may be used as the evaluation value on a pixel basis. In order to simplify the process, it may be determined that the spectral data R1 coincides with the spectral data R2 when the difference between the spectral data R1 and the spectral data R2 is equal to or less than a threshold value set in advance. The wavelength range in the expression (14) is 380 to 780 nm. Note that the value E may be calculated within an arbitrary wavelength range.
E
=
1
n
∑
i
=
380
780
(
R
1
λ
i
-
R
2
λ
i
)
2
(14), where, λ is the wavelength, and n is the number of sample data in the wavelength direction.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Kimura, Kadoi, and Kikuchi as within the same field of image processing using spectral characteristics, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, in that by substituting the region detection in the device of the combination of Kimura and Kadoi with the spectral based region detection of Kikuchi, you effectively reduce the need to perform the template matching and thus can significantly reduce processing. Furthermore, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that certain objects reflect certain wavelengths/spectrums of light, and thus detecting said objects via wavelength could provide a more accurate detection of the object. This is specifically relevant in applications wherein the object of interest may be partially obfuscated or not resemble a standard template but share a common characteristic with other objects (e.g., plants are generally green, basketballs are generally orange, etc.). Therefore, another motivation would be to improve the accuracy of the detection by detecting object for which template matching would not function. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the device of the combination of Kimura with the selection from previously measured light source spectral information of Kadoi, and further combined the device of the combination of Kimura and Kadoi with the spectral based region detection of Kikuchi, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi, and the spectral based region detection of Kikuchi to obtain the invention as specified in claim 8.
24. Regarding Claim 18, a combination of Kimura and Kadoi teaches the non-transitory computer readable storage medium of claim 12. The claim language is analogous to claim 8. Rejections analogous to claim 8 are further applicable to claim 18 in view of the non-transitory computer readable storage medium of Kimura. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the non-transitory computer readable storage medium of Kimura with the selection from previously measured light source spectral information of Kadoi, and the spectral based region detection of Kikuchi to obtain the invention as specified in claim 18.
25. Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over JP-2017215851-A to Kimura, in view of U.S. Publication No. 2021/0274102 to Kadoi, and further in view of “Strategy for the Development of Smart NDVI Camera System for Outdoor Plant Detection and Agricultural Embedded Systems” to Dworak et al. (hereinafter Dworak).
26. Regarding Claim 9, a combination of Kimura and Kadoi teaches the device of claim 1. Kimura discloses wherein the operations further comprise: detecting the specific region ([par. 0046, ln. 1-2], [Fig. 5(b)], [par. 0051, ln. 1-8], [par. 0054, ln. 1-10]) {with a plant as the specific object}. Kimura does not specifically disclose a plant is the specific object detected. Likewise, Kadoi does not specifically disclose wherein a plant is the specific object.
However, Dworak teaches a plant as the specific object ([pg. 1524, Introduction, par. 1, ln. 1-15] “…The coverage level or plant counts are typical values used to control a field sprayer. The coverage level can be calculated from the NDVI image of the local field conditions [2], and the plant number in a local scene can be estimated using an additional algorithm. The NDVI is a parameter used to separate vital plant pixels from soil pixels in an image or to separate vital from non-vital plants. The NIR reflection is high for vital plants and low for soil; plants absorb more light with red wavelengths, from 620 nm to 660 nm, than soil [6,7]. Nutrient supply of the plants influences absorption through chlorophyll activity in the transition band from red to NIR (660 nm to 740 nm) and thereby corresponds to the stress of the plant [8]. Reflections in the wavelength spectrum below 740 nm are higher for plants than for soil. Hence, NIR wavebands below 780 nm are commonly used for the NDVI. Thus, the difference between NIR and red is high for plants.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Kimura, Kadoi, and Dworak as within the same field of image processing using spectral characteristics, and as analogous to the claimed invention. The motivation to combine the device of the combination of Kimura and Kadoi with the plant as a specific object as taught in Dworak would have been obvious to one of ordinary skill in the art, in that by using the device of the combination of Kimura and Kadoi on plant detection analogous to Dworak, you could effectively remove ambient light caused by outside light sources (e.g., flash or sunlight), and therefore improve the ability to determine coverage level of a plant, and furthermore that taking images of a plant as taught in Dworak offers a real-world application for the device of the combination of Kimura and Kadoi. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the device of Kimura with the selection from previously measured light source spectral information of Kadoi, and further combined the device of the combination of Kimura and Kadoi with the plant as a specific object as taught in Dworak, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the device of Kimura with the selection from previously measured light source spectral information of Kadoi, and the plant as a specific object as taught in Dworak to obtain the invention as specified in claim 9.
27. Regarding Claim 19, a combination of Kimura and Kadoi teaches the non-transitory computer readable storage medium of claim 12. The claim language is analogous to claim 9. Rejections analogous to claim 9 are further applicable to claim 19 in view of the non-transitory computer-readable storage medium of Kimura. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the non-transitory computer-readable storage medium of Kimura with the selection from previously measured light source spectral information of Kadoi, and the plant as a specific object as taught in Dworak to obtain the invention as specified in claim 19.
Conclusion
28. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights