Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant first mentions that independent claims 1, 19 and 20 have been amended to overcome the 112(b) rejection and 103 rejection by newly featured limitations including “wherein, in the processing of the texture data, in a case when the size of the set region is larger than the predetermined size of the texture image, tiling is performed using the texture image so as to accord with the size of the set region, as a result of tiling using the texture image, the set region is covered by repeatedly arranged identical texture images, and clipping is performed on a texture image in which the tiling has been performed after tiling using the texture image”, examiner agrees with the 112(b) fix and agrees with the 103 because of the underlined part of the limitation. The amended claims did affect the 103 rejection, however only the underlined part of the claim limitation, the rest of the limitation was already rejected in cancelled claims 4 and 5. Examiner has added new cited prior art Lebovitz to account for the added limitation. All of the claims stay rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims are 1, 6, 7, 13, 14, 18, 19, 20, 22, 23, 25 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto et. al, hereafter Yamamoto (EP Application No. 0961230 A2) in view of Oh et. al. (US Publication No. 20040095357 A1), Wang et. al. (Wang, Ruixing, et al. "Underexposed photo enhancement using deep illumination estimation." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.), also in view of Hu et. al., hereafter Hu (US Publication No. 20190355119 A1), Kuwabara et. al. (US Publication No. 20240394905 A1), Lebovitz (US Pub. No. 20150325044 A1) and Hurley et. al. (WO Pub. No. 2012118868-A2)
As per claim 1, Yamamoto teaches “a method comprising:
“setting a region in which decoration processing is to be applied on an image according to decoration data;” (Examiner interprets “decoration data” as “mosaic image”; “setting a region” as “divided into a plurality of areas”. See page 2 paragraph 4 “To generate a mosaic image, an original image is divided into a plurality of areas, and in each of the areas, a texture image which will construct the mosaic image is pasted.” Yamamoto)
“holding… texture data representing a texture image in a predetermined size”, it also teaches “acquiring the texture data” (The predetermined size is p’ x q’ pixels. Page 10 Paragraph 91 “In step S33, average density tk for each of R, G and B is calculated with respect to each of the P sheets of texture images 203. This is calculated by the following equation, assuming that the size of each texture image is p' x q' (pixels). Σ indicates the sum of color components of all pixels included in a texture image.” Yamamoto)
“and illumination data representing brightness contrast in a region in a predetermined size”, it also teaches “acquiring the illumination data”(Examiner interprets “illumination data” as “average luminance”. The average luminance is acquired from the same texture image as the quotation from the former limitation, therefore is has a “predetermined size”, which is p’ x q’ pixels.) ( Page 2 Paragraph 15 “When comparing a divided area (tile area) of an original image with each texture image, average luminance or average density per pixel is calculated with respect to three primary colors: red (R), green (G) and blue (B) for each texture image and a divided area subjected to processing, and a texture image having a minimum difference in the average luminance or average density is selected as a texture image of the subject divided area. Yamamoto)
“performing processing of at least one of tiling or clipping on the acquired texture data so as to accord with a size of the set region… of the acquired texture data” (Page 5 Paragraph 35. This is interpreted as tiling, therefore it teaches at least one of the limitations. “When pasting the selected texture image 203 on the corresponding tile area, if the size of the texture image does not match the size of the tile area, size changing process is performed so that the size of the selected texture image matches the size of the tile area.” Yamamoto)
“performing processing of either enlarging or reducing a size of a region of…” and “…so as to accord with the size of the set region;” (A size changing process is performed on it to have the same size as the “set region” (tile area). A size changing process cover the interpretation of “enlarging or reducing”. Page 5 Paragraph 35 “When pasting the selected texture image 203 on the corresponding tile area, if the size of the texture image does not match the size of the tile area, size changing process is performed so that the size of the selected texture image matches the size of the tile area.”. “Size changing process so that the size of the selected texture image matches the size of the tile area” covers both enlarging and reducing.” Yamamoto)
“generating the decoration data from the processed texture data… and is performed by the processing of at least one of tiling or clipping… and the processed illumination data… ; and applying the decoration data to the set region.” (Yamamoto teaches on page 2 paragraph 1 that it generates a mosaic image “decoration data” based on texture and paragraph 15 teaches that the texture data also includes illumination data “average luminance”. Fig 1 shows that it is applied to the whole image (the set region in this case) Page 2 Paragraph 1 “The present invention relates to an image processing method and apparatus thereto for generating a mosaic image by combining a plurality of texture images.” Page 2 Paragraph 15 “When comparing a divided area (tile area) of an original image with each texture image, average luminance or average density per pixel is calculated with respect to three primary colors: red (R), green (G) and blue (B) for each texture image and a divided area subjected to processing, and a texture image having a minimum difference in the average luminance or average density is selected as a texture image of the subject divided area.” Yamamoto)
“wherein, in the processing of the texture data, in a case when the size of the set region is larger than the predetermined size of the texture image, tiling is performed using the texture image so as to accord with the size of the set region, as a result of tiling using the texture image…” (See Fig. 1, let the image 201 be the size of the set region and 203 the predetermined size of the texture image, tiling is performed using 203 to obtain the size of 201 in 202. See also Page 5 Paragraph 35. Tiling is at least performed when the size of the set region is larger than the predetermined size of the texture image. The paragraph covers both situations, when a region is larger or smaller, tiling is always performed so as to match sizes of the regions. This is interpreted as tiling, therefore it teaches at least one of the limitations. “When pasting the selected texture image 203 on the corresponding tile area, if the size of the texture image does not match the size of the tile area, size changing process is performed so that the size of the selected texture image matches the size of the tile area.” A size changing process is performed on it to have the same size as the “set region” (tile area). A size changing process cover the interpretation of “enlarging or reducing”. Page 5 Paragraph 35 “When pasting the selected texture image 203 on the corresponding tile area, if the size of the texture image does not match the size of the tile area, size changing process is performed so that the size of the selected texture image matches the size of the tile area.”. “Size changing process so that the size of the selected texture image matches the size of the tile area” covers both enlarging and reducing.” Yamamoto)
Yamamoto does not teach “holding, in a separate state from each other, texture data… and illumination data…”, “texture data held in the separate state from the illumination data” and “illumination data held in the separate state from the texture data”, “without performing processing of either enlarging or reducing a size of a region of the acquired texture data”, “performing processing of either enlarging or reducing a size of a region of the acquired illumination data”, “without performing processing of either tiling or clipping on the acquired… data”.
On the last limitation, Yamamoto also does not teach “which is held in a separate state from the illumination data…”, “ without being performed by the processing of either enlarging or reducing the size of the region of the acquired texture data ” and “…which is held in a separated state from the texture data”, “and is performed by the processing of either enlarging or reducing the size of the region of the acquired illumination data” and “without being performed by the processing of either tiling or clipping on the acquired… data” and “…the set region is covered by repeatedly arranged identical texture images, and clipping is performed on a texture image in which the tiling has been performed after tiling using the texture image.”
Oh teaches “holding, in a separate state from each other, texture data… and illumination data…”, “texture data held in the separate state from the illumination data” and “illumination data held in the separate state from the texture data”, and on the last limitation “which is held in a separate state from the illumination data…”, “…which is held in a separated state from the texture data” and generally “illumination data” (See abstract, paragraphs 132 and fig. 19. Texture data and illumination data are held separately from the other.)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto with the teachings of Oh to separately hold illumination data and texture data. The modification would have been motivated by the desire to have better usability and be able to easily edit either the texture data or illumination data as suggested by Oh (See all of paragraph 132 “… It is a non-linear image-processing filter that decouples the lighting effects from uniformly textured images, and thus enables the user to easily edit either the materials or the lighting characteristics of the input image…” Oh)
Wang teaches “performing processing of either enlarging or reducing a size of a region of the acquired illumination data” and on the last limitation “… and is performed by the processing of either enlarging or reducing the size of the region of the acquired illumination data” and generally “acquired illumination data” (See fig. 3 and paragraph 2 on page 4 Efficient Runtime. It teaches enlargement of the illumination data and the illumination data itself. Examiner interprets a size of a region as the whole region. Wang)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto and Oh with the teachings of Wang to enlarge or reduce illumination data. The modification would have been motivated by the desire to have a higher quality image by upsampling, which increases resolution, as suggested by Wang (See page 4 paragraph 2 Efficient Runtime. “We learn the local and global features for predicting the image-to-illumination mapping in low resolution, and perform bilateral grid based upsampling [8, 7, 12, 13] to enlarge the low-res prediction to the full resolution; see Figure 3. Hence, most network computation is done in low-res domain, enabling real-time processing of high-resolution images.”).
Hu teaches “without performing processing of either enlarging or reducing a size of a region of the acquired texture data” and on the last limitation“ without being performed by the processing of either enlarging or reducing the size of the region of the acquired texture data ” and (See paragraph 77 and fig. 2. “[0077] With the background and the ROIs identified and segmented, the ROIs are rearranged to pack the ROIs closer together, as in a step 410. Herein rearranging an individual ROI means performing one or more geometric operations without resizing the individual ROI. Furthermore, each geometric operation that is performed does not result in a loss of information carried by the individual ROI and necessary for cell classification. Relocating the individual ROI, as mentioned above in explaining FIG. 2, is one such geometric operation and is an essential operation in compacting the testing image.” On paragraph 77 it shows that the process of resizing (enlarging or reducing) is not performed on the region of interest (ROI) and as seen in fig. 2 it is on texture data. )
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto, Oh, Wang with the teachings of Hu to perform tiling or clipping without enlarging or reducing a region of the texture data. The modification would have been motivated by the desire to have better data integrity and not lose necessary information from the texture, as suggested by, Hu (See fig. 2 and paragraph 77. “[0077] With the background and the ROIs identified and segmented, the ROIs are rearranged to pack the ROIs closer together, as in a step 410. Herein rearranging an individual ROI means performing one or more geometric operations without resizing the individual ROI. Furthermore, each geometric operation that is performed does not result in a loss of information carried by the individual ROI and necessary for cell classification. Relocating the individual ROI, as mentioned above in explaining FIG. 2, is one such geometric operation and is an essential operation in compacting the testing image.” Hu).
Kuwabara teaches “enlarging or reducing a size of a region of the acquired…data so as to accord with the sizer of the set region, without performing processing of either tiling or clipping on the acquired… data” and on the last limitation “enlarging or reducing a size of the region of the acquired…data… without being performed by the processing of either tiling or clipping on the acquired… data” (See paragraphs 146, 179 and 217 “[0146]… Furthermore, the clipping unit 200 may generate a recognition image 1104b by reducing the entire captured image 1100N to the predetermined size without clipping the captured image 1100N. In this case, the clipping unit 200 can add the position information delivered from the detection unit 201 to the recognition image 1104b. ” Paragraphs 96, 152 and 156-157 also shows that the pixels contain illumination data, so the process implicitly affects the illumination data. Kuwabara)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto, Oh, Wang, and Hu with the teachings of Kuwabara to resize illumination data without performing clipping. The modification would have been motivated by the desire to have higher processing speed, as suggested by, Kuwabara (See paragraphs 217 and 4-5 “[0005] The present disclosure provides an image processing device and an image processing method which enable execution of recognition processing at a higher speed.”).
Lebovitz teaches “the set region is covered by repeatedly arranged identical texture images,” (See paragraphs 39, 68 and 6 “[0039] “Image tile” means an image that is meant to be repeated in a pattern to generate a surface texture;” “[0068] In one example, according to the techniques described herein, however, an image tile is to be repeated across the surface of the mesh as a pattern…” “[0006]… scaling and repeating the image tile so that the entire bounding box is covered considering the differences in scale between the new texture image tile in image space and the parameterization scale as it relates to R3; converting each resulting scaled, tiled, texture image to HSV space; and compositing the V channel of the scaled, tiled texture image in HSV color space with the shading image, then converting the result back to RGB color space, to generate the final texture map image.” See also paragraphs 122, 113, 75, 74 and fig. 4, fig. 9, fig. 10. “[0075]… Since UV locations outside of the range [0, 1) correspond to repeats of the image tile, the process repeats the image tile vertically and horizontally an integer number of times to ensure all image space locations that correspond to UV locations of vertices in the new parameterization lie on a pixel of the unshaded image.” Lebovitz )
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu, and Kuwabara with the teachings of Lebovitz to cover the set region with repeated identical texture images. The modification would have been motivated by the desire to easily switch patterns and produce texture mapped models without acquiring additional and unnecessary photography and scanning, therefore it is an improvement, as suggested by Lebovitz (See paragraph 59 “[0059] This disclosure describes techniques to easily switch patterned images on a photorealistic texture map applied to an arbitrary 3D scanned mesh… the new texture image uses shading from the original texture map applied to a selected image tile that is scaled and repeated appropriately according to the real-world 2D size to which the image tile corresponds. The result is that photorealistic texture-mapped 3D models that are photographed can be changed to display any material pattern desired without physically changing the real-world object and acquiring additional photography and scanning.” Lebovitz)
Hurley teaches “and clipping is performed on a texture image in which the tiling has been performed after tiling using the texture image.” ( Hurley in paragraph 84 teaches the tiling of an image (“tiles are stitched to produce a complete image” Hurley). In paragraph 85, Hurley teaches that the images are clipped to produce square images. The images used in Hurley have texture, since the images are of petrographic (rock) as suggested by the abstract. “[0084] In block 11 14, the image is pre-processed and enhanced. The goal of this phase is to prepare the raw image for image analysis. Raw image tiles of 512 by 512 pixels in size, for example, are stitched to produce the complete image.” Hurley)
(“[0085] The images are taken through a number of steps. Clipping and rotation are applied to produce square images, according to some embodiments.” Hurley)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu, Kuwabara and Lebovitz with the teachings of Hurley to use clipping on a tiled image. The modification would have been motivated by the desire of producing a uniform image with the same size (square image) with no irregularities as suggested by Hurley (“[0106] The images produced were taken through a number of steps. Clipping and rotation was applied to produce square images. Manual adjustments in Photoshop or Paint.Net were applied in order to remove some irregularities. Image normalization is commonly needed to adjust brightness of individual z-steps to match a chosen standard.” Hurley)
Claims 19 and 20 are rejected under the same analysis as claim 1. (See page 5-6 paragraph 40, it shows processor, memory and readable media. Yamamoto)
As per claim 7, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1”, however Yamamoto also teaches “wherein, in the processing to be performed on the illumination data, in a case when the size of the set region is larger than the predetermined size of the region of the brightness contrast, enlargement of the region of the illumination data is performed.” (Examiner interprets the “size of the set region” as “size of the tile area” and “predetermined size of the region of the illumination data” as “the size of the texture image” from Yamamoto. The texture image already includes illumination data and a predetermined size as explained in claim 1. In page 4 line 29 it is interpreted that the reference teaches both cases in which one (“set region” and “predetermined size of the region”) is larger or smaller than the other and vice versa. It then performs a size conversion of the “texture image”, which includes illumination data, to a bigger or smaller size, therefore it also teaches the “enlargement of the region of the illumination data”. Examiner interprets “enlargement of the region of the illumination data” as any change in size of the illumination data. Page 4 Line 29, “In a case where a texture image is pasted on a tile area but the size of the texture image does not match the size of the tile area, the size of the texture image needs to be converted to match the size of the tile area.” Yamamoto)
As per claim 6, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1, wherein in the processing to be performed on the texture data, in a case when the size of the set region is smaller than the predetermined size of the texture image,” as explained in the claim 7 rejection, however Yamamoto in view of Oh, Wang, Hu and Kuwabaraand Lebovitz does not teach “clipping is performed on the texture image”
Hurley teaches “clipping is performed on the texture image.” (In paragraph 85, Hurley teaches that the images are clipped to produce square images. The images used in Hurley have texture, since the images are of petrographic (rock) as suggested by the abstract.)
(“[0085] The images are taken through a number of steps. Clipping and rotation are applied to produce square images, according to some embodiments.” Hurley)
As per claim 13, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1, wherein, in the acquiring of the illumination data,”, however Yamamoto also teaches “the illumination data is acquired based on the size of the set region.” (The texture image has an “average luminance” which is calculated, and a size changing process is performed on it to have the same size as the “set region” (tile area), therefore the illumination data is acquired based on the size of the region. Page 5 Paragraph 35 “When pasting the selected texture image 203 on the corresponding tile area, if the size of the texture image does not match the size of the tile area, size changing process is performed so that the size of the selected texture image matches the size of the tile area.” Yamamoto)
As per claim 14, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1”, however Yamamoto also teaches “further comprising, in a case when the set region includes a plurality of objects, extracting a plurality of regions from the set region, wherein each of the extracted plurality of regions is processed as a set region.” (See Fig.3, 202 is interpreted as the set region and each tile area 203 is an extracted object and set region, Yamamoto)
As per claim 18, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1”, however Yamamoto also teaches “further comprising causing a printing apparatus to print data to which the decoration data has been applied.” (In Page 9 Paragraph 85 the mosaic imager ( data to which the decoration data has been applied) is printed by a printer “In the hard disk 103, a plurality (P) of texture images 203 which become components of a mosaic image are stored. M x N sheets of images, selected from the texture images 203 according to a program which will be described later, are arranged as shown in Figs. 1 and 13, M sheets in the horizontal direction and N sheets in the vertical direction, to form a mosaic image. A mosaic image formed in this manner is stored as an image file in the hard disk 103, or displayed in the display unit 105, or printed by the printer 109.” Yamamoto)
As per claim 22, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1,” wherein, in the processing to be performed on the illumination data, in a case when the size of the set region is smaller than the predetermined size of the region of the brightness contrast, reduction of the region of the illumination data is performed.” (See page 4 paragraph 26 and page 5 paragraph 35. When size changing is performed, reduction of the illumination data is performed since it needs to accord to a size that is smaller. Yamamoto)
As per claim 23, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “The method according to claim 1, wherein, as a result of tiling using the texture image, the identical texture images are arranged adjacent to each other.” (See paragraphs 39, 68 and 6 “[0039] “Image tile” means an image that is meant to be repeated in a pattern to generate a surface texture;” “[0068] In one example, according to the techniques described herein, however, an image tile is to be repeated across the surface of the mesh as a pattern…” “[0006]… scaling and repeating the image tile so that the entire bounding box is covered considering the differences in scale between the new texture image tile in image space and the parameterization scale as it relates to R3; converting each resulting scaled, tiled, texture image to HSV space; and compositing the V channel of the scaled, tiled texture image in HSV color space with the shading image, then converting the result back to RGB color space, to generate the final texture map image.” See also paragraphs 122, 113, 75, 74 and fig. 4, fig. 9, fig. 10. “[0075]… Since UV locations outside of the range [0, 1) correspond to repeats of the image tile, the process repeats the image tile vertically and horizontally an integer number of times to ensure all image space locations that correspond to UV locations of vertices in the new parameterization lie on a pixel of the unshaded image.” Lebovitz )
As per claim 25, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley teaches “The method according to claim 1, wherein the setting of the region is performed via a user interface screen of a decoration application for adding a decoration effect to an image by combining into image information texture information.” (See page 14 paragraph 12 and fig. 21-24 “[0146] Fig. 21 shows an example of a user interface displayed for changing the value α. The user interface shown in Fig. 21 is displayed in the display unit 105. In Fig. 21, reference numeral 401 denotes a display area of the first image 201; and 402, a display area of the third image. Reference numeral 403 denotes a start button instructing a start of the process; and 404, a slider for changing the value α. Note that an area for displaying the second image 202 may be provided above this screen so that an operator can refer to the second image 202 when setting the value α.” See also page 15 paragraph 6 and “[0153]-[0159]” “In step S803, a window 61 pops up in the display unit 105 as shown in Fig. 24. The operator specifies an appropriate value α for the tile area 54 by using the window 61. At this stage, the window 61 is displayed at the position so as not to hide the tile area 54 for operative convenience… In the window 61, reference numeral 62 denotes a display area for displaying a texture image 203 originally selected for the tile area 54. Reference numeral 63 denotes a display area for displaying a corrected texture image, obtained by correcting the original image 201 with the value α by the process to be described later” and “[0153] As described above, the seventh embodiment is so structured that the value α can be set interactively by using the input unit such as a mouse or the like so that the user can easily produce a mosaic image by synthesizing the first image and second image as desired.” Therefore it also utilizes it for combining texture and image information. See also page 17 paragraphs 8-12 “[0176]-[0180]” “When the process is completed for the p x q pixels, the control proceeds to step S919 where the value α is calculated by the following equation. α = A / √(e / (pxq)). Herein, "A" is a constant which determines the amount of mixing the images, and may be determined empirically or may be set arbitrarily by a user. Moreover, e/(pxq) expresses an average density difference for each color component in the entire pixels of the tile area (texture image).” Yamamoto) (See also Lebovitz paragraphs 78 and 67-69. Lebovitz)
As per claim 26, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley teaches “The method according to claim 25, wherein the decoration application is an application to cause an inkjet printer to print an image” (See page 6 first paragraph “…Reference numeral 109 denotes a printer, such as a color ink-jet printer or a color electrophotographic printer.” See also page 9 paragraph [0077] “In step S601 in Fig. 10, in a case of printing a produced mosaic image by the printer 109, a message inquiring whether the mosaic image is to be printed in color or monochrome is displayed in the display unit 105. In step S602, an operator who reads the displayed message selects a desired print mode: color or monochrome, by using the keyboard 104 or mouse 104a. In step S603, the process in step S604 is performed if the selected print mode is color printing, while the process in step S605 is performed if the selected print mode is monochrome printing.” Yamamoto), however Lebovitz teaches “print an image including an object to which a decoration pattern has been applied.” (See paragraphs 5, 78 “0078] As described above, the techniques described herein may be used to change the surface texture of a target virtual object. FIG. 1 is a flow diagram depicting an example process of changing the surface texture of a target virtual object according to the example techniques described herein. It should be noted that a computing system, for example, computing system 1200, may be configured to perform one or more steps of process 100. Further, a computing system may be configured to enable a user to perform one or more of the steps of process 100 (e.g., by providing a graphical user interface enabling a user to perform a step).” Figs. 3, 4 10 also show a printed image which includes an object in which a texture pattern has been applied. See also paragraphs 13, 14, 20 and 210 “[0020] FIG. 10 illustrates an example of changing fabric on 3D models of furniture.” See also paragraph 72 “The process is described in further detail below. The result is that the image tile is repeated in a consistent orientation and at a consistent scale to create the effect of a repeated tile across the surface. Taking the previous example of a couch, the polka dot pattern is repeated as if a fabric is rolled out across a surface, and therefore cannot change directions without bunching up the flat fabric, which is undesirable. Furthermore, its scale remains consistent since the pattern would be printed on a fabric at a constant size.” It is motivated by the already presented reasons. Lebovitz)
Claims 2 is rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz, Hurley and further in view of Inoue Yuichi, hereafter Inoue (JP Publication No. 200993287 A)
As per claim 2, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1, wherein in the processing to be performed on the texture data,”, however Yamamoto in view of Oh, Wang, Hu and Kuwabara does not directly teach “a frequency characteristic of the texture image is not changed”.
Inoue teaches “a frequency characteristic of the texture image is not changed”. (Examiner interprets “frequency characteristic” as how the texture or pattern is perceived, same texture before and after) (Page 6 Paragraph 3 Inoue reproduces the texture of the texture, therefore its frequency characteristic does not change. “In S7, the texture data processing unit 21 performs a process of reflecting the surface reflection and shadow generated by the light source on the texture data read in S4 based on the light source information acquired in S6. This processing is performed using a three-dimensional object that reproduces the texture of the texture (transparency, reflectance, refractive index, metallic feeling, etc.).” Inoue)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley with the teachings of Inoue to not change the frequency characteristic of the texture. The modification would have been motivated by the desire of correctly reproducing how a texture is perceived as suggested by Inoue (Page 4 Paragraph 4 “For example, when a material with a metallic feeling such as gold or silver is used as the texture, the metal reflection caused by the light source at the viewing place cannot be faithfully reproduced, and only a printed image lacking in reality may be obtained.” Inoue)
Claims 8, 9, 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz, Hurley and further in view of Nakamura et. al, hereafter Nakamura (US. Publication No. 20180039852 A1).
As per claim 8, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 7, wherein in the processing to be performed on the illumination data, in a case when the size of the set region is larger than the predetermined size of the region of the brightness contrast” and “enlargement of the region of the illumination data is performed.” Yamamoto does not teach “and a change of the brightness contrast represented by the illumination data is less than a reference”
Nakamura teaches “and a change of the brightness contrast represented by the illumination data is less than a reference” (Examiner interprets “a change of the brightness contrast represented by the illumination data” as “luminance variation” and “less than a reference” as “not larger than the threshold value dI” from Nakamura. The threshold value is the reference. Paragraph 155 “If the result of the determination by the luminance variation comparison unit 231 indicates that the luminance variation dI is “not larger than the threshold value TI”, the pixel in question is determined to belong to a background region.” Nakamura)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley with the teachings of Nakamura to enlarge the region of the illumination data based on a change of brightness contrast that is less than a reference. The modification would have been motivated by the desire of having better accuracy in terms of processing the data in the region as suggested by Nakamura (Paragraph 357 “among the images used for the calculation of the luminance variation dI, is not larger than the threshold value, then the target value related to the illumination condition is calculated such that the light emitting intensity of the above-mentioned illumination condition (illumination condition with the smaller light emitting intensity) is enlarged (that is, the target value of the light emitting intensity is enlarged)… Accordingly, it is possible to obtain images suitable for the extraction of the subject region regardless of the change in the ambient light, thereby improving the accuracy in the extraction of the subject region.” Nakamura )
As per claim 9, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley and Nakamura already teaches “the method according to claim 8, wherein in the processing to be performed on the illumination data, in a case when the size of the set region is larger than the predetermined size of the region of the brightness contrast” and “tiling using the illumination data is performed instead of enlargement of the region of the illumination data being performed.” however Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley does not teach “and the change of the brightness contrast represented by the illumination data is greater than the reference”.
Nakamura also teaches “and the change of the brightness contrast represented by the illumination data is greater than the reference” (Examiner interprets “a change of the brightness contrast represented by the illumination data” as “luminance variation” and “greater than a reference” as “ larger than the threshold value dI” from Nakamura. The threshold value is the reference. Paragraph 156 “ If the result of the determination by the luminance variation comparison unit 231 indicates that the luminance variation dI is “larger than the threshold value TI”, and the result of the determination by the texture variation comparison unit 232 indicates that the texture variation dF is “not larger than the threshold value TF”, the pixel in question is determined to belong to a subject region.” Nakamura)
As per claim 10, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz, Hurley and Nakamura already teaches “the method according to claim 9, wherein, in the processing of the illumination data”,
however Hurley teaches “after tiling using the illumination data is performed, clipping is performed on a region of illumination data in which the tiling has been performed, so as to accord with the size of the set region.” ( Hurley in paragraph 84 teaches the tiling of an image (“tiles are stitched to produce a complete image”). In paragraph 85, Hurley teaches that the images are clipped to produce square images (Examiner interprets the “square images” as ”the size of the set region“). Examiner also interprets that the images already have illumination data, therefore they are interchangeable. “[0084] In block 11 14, the image is pre-processed and enhanced. The goal of this phase is to prepare the raw image for image analysis. Raw image tiles of 512 by 512 pixels in size, for example, are stitched to produce the complete image.” Hurley)
(“[0085] The images are taken through a number of steps. Clipping and rotation are applied to produce square images, according to some embodiments.” Hurley )
As per claim 11, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 8, wherein in the processing of the illumination data, in a case when the size of the set region is larger than the predetermined size of the region of the brightness contrast”, however Yamamoto in view of Oh, Wang, Hu Kuwabara, Lebovitz and Hurley does not teach “and the change of the brightness contrast represented by the illumination data is greater than the reference, supplementation of a pixel in a periphery of the region of illumination data is performed instead of enlargement of the region of the illumination data being performed.”
Nakamura already teaches “ and the change of the brightness contrast represented by the illumination data is greater than the reference” in claim 8, however Nakamura also teaches “supplementation of a pixel in a periphery of the region of illumination data is performed instead of enlargement of the region of the illumination data being performed.” (Examiner interprets “supplementation of a pixel in a periphery of the region of illumination data” as “the pixel in question is determined to belong to a subject region” from Nakamura. Supplementation of a pixel is interpreted as including a pixel. Examiner also interprets that the “periphery of the region” is covered by the “subject region” in Nakamura. Paragraph 156 “[0156] If the result of the determination by the luminance variation comparison unit 231 indicates that the luminance variation dI is “larger than the threshold value TI”, and the result of the determination by the texture variation comparison unit 232 indicates that the texture variation dF is “not larger than the threshold value TF”, the pixel in question is determined to belong to a subject region.” Nakamura)
Claims 12 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley and further in view of Ichihashi (US Publication No. 20180054547 A1 ).
As per claim 12, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 1”, however Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley does not teach “further comprising performing adjustment of a brightness contrast on the processed illumination data based on a skewness of a luminance histogram of the texture image.”
Ichihashi teaches “further comprising: performing adjustment of a brightness contrast on the processed illumination data based on a skewness of a luminance histogram of the texture image.” (Ichihashi teaches that “lightness of the pixels” (brightness contrast of the illumination data) is adjusted based on a “lightness histogram” (luminance histogram) of the image data (interpreted as a texture image since it includes “glossiness” i.e. a texture). Then in paragraph 69, for the 3rd embodiment, Ichihashi teaches that is uses the skewness of a lightness histogram to adjust the glossiness of the image. Abstract “An image processing apparatus and a method controlling the same extract an object included in image data and extract a highlight region of the object. A region including the highlight region and the pixels around the highlight region are obtained in the object. The lightness histogram of the pixels included in the region is generated, and the lightness of the pixels except the highlight region is adjusted based on the generated histogram.” Ichihashi.)
(Paragraph 69 “As has been described above, according to the third embodiment, the plurality of lightness adjustment lookup tables are generated, and the lookup table having the highest skewness of the lightness histogram is selected from the plurality of lookup tables and used to adjust the glossiness of the image data. Ichihashi)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley with the teachings of Ichihashi to adjust the brightness contrast based on skewness of a histogram. The modification would have been motivated by the desire to correctly represent the texture (glossiness) by determining whether the skewness is positive or negative as suggested by Ichihashi (“[0046] The process then advances to step S306, and the CPU 101 determines whether or not the skewness in step S305 is positive, that is, whether or not the object has glossiness. In this case, if the skewness is negative, the object is considered not to have glossiness, thereby directly ending the process. If the image is determined to have glossiness, the image data can be adjusted to increase the glossiness.” Ichihashi)
As per claim 24, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “The method according to claim 1…” however Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley does not teach “wherein a skewness of a histogram of luminance in the texture image is larger on a positive side.”
Ichihashi teaches “wherein a skewness of a histogram of luminance in the texture image is larger on a positive side.” (Ichihashi teaches that “lightness of the pixels” (brightness contrast of the illumination data) is adjusted based on a “lightness histogram” (luminance histogram) of the image data (interpreted as a texture image since it includes “glossiness” i.e. a texture). Then in paragraph 69, for the 3rd embodiment, Ichihashi teaches that is uses the skewness of a lightness histogram to adjust the glossiness of the image. Abstract “An image processing apparatus and a method controlling the same extract an object included in image data and extract a highlight region of the object. A region including the highlight region and the pixels around the highlight region are obtained in the object. The lightness histogram of the pixels included in the region is generated, and the lightness of the pixels except the highlight region is adjusted based on the generated histogram.” Ichihashi. Paragraph 69 “As has been described above, according to the third embodiment, the plurality of lightness adjustment lookup tables are generated, and the lookup table having the highest skewness of the lightness histogram is selected from the plurality of lookup tables and used to adjust the glossiness of the image data.” See also paragraphs 44-55 “[0046] The process then advances to step S306, and the CPU 101 determines whether or not the skewness in step S305 is positive, that is, whether or not the object has glossiness. In this case, if the skewness is negative, the object is considered not to have glossiness, thereby directly ending the process. If the image is determined to have glossiness, the image data can be adjusted to increase the glossiness. For this reason, the process advances to step S307.” Therefore when there is glossiness, skewness is larger on the positive side, if it were more negative, there would be no glossiness. Ichihashi)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley with the teachings of Ichihashi to include the use of skewness in a histogram of luminance in the texture image and show a case where the skewness is larger. The modification would have been motivated by the desire to correctly represent the texture (glossiness) by determining whether the skewness is positive or negative as suggested by Ichihashi (See paragraphs 68, 4, 5, 24 and 46“[0046] The process then advances to step S306, and the CPU 101 determines whether or not the skewness in step S305 is positive, that is, whether or not the object has glossiness. In this case, if the skewness is negative, the object is considered not to have glossiness, thereby directly ending the process. If the image is determined to have glossiness, the image data can be adjusted to increase the glossiness.” Ichihashi)
Claims 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz, Hurley and further in view of Taylor (US Publication No. 20180101510 A1 )
As per claim 15, Yamamoto in view of Oh, Wang, Hu , Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 14”, however Yamamoto in view of Oh, Wang, Hu , Kuwabara, Lebovitz and Hurley does not completely teach “wherein, in the extracting of the plurality of regions, extracting, as the plurality of regions, regions respectively corresponding to the plurality of objects.”
Taylor teaches “wherein in the extracting of the plurality of regions, extracting, as the plurality of regions, regions respectively corresponding to the plurality of objects.” (See Fig. 3, Examiner interprets each character as both a region and object. Taylor)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu , Kuwabara, Lebovitz and Hurley with the teachings of Taylor to extract the region corresponding to an object within a region. The modification would have been motivated by the desire to correctly and consistently apply an effect to all possible regions as suggested by Taylor (“[0039] FIG. 3 illustrates an example of graphical effects of advanced ink effects being applied to strokes by applying an ink effect definition of an image file. The geometries 310 for strokes are illustrated via dashed lines, which are applied to an image definition 320 to create an effect mask 330 for each stroke. By creating an effect mask 330 rather than pushing a spectral line along a stroke, greater effect consistency across strokes and ink objects may be achieved.” Taylor)
As per claim 16, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 14”, however Yamamoto in view of Oh, Wang, Hu , Kuwabara, Lebovitz and Hurley does not completely teach “wherein the extracted regions include a plurality of objects.”. Lebovitz also teaches “wherein the extracted regions include a plurality of objects” (See abstract, para 57, 59 and 78. Lebovitz)
Taylor teaches “wherein the extracted regions include a plurality of objects.” (See Fig. 3, the regions include a plurality of objects (characters). Taylor)
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to combine the teachings of Yamamoto in view of Oh, Wang, Hu and Kuwabara with the teachings of Taylor to extract the region corresponding to an object within a region. The modification would have been motivated by the desire to correctly and consistently apply an effect to all possible regions as suggested by Taylor (“[0039] FIG. 3 illustrates an example of graphical effects of advanced ink effects being applied to strokes by applying an ink effect definition of an image file. The geometries 310 for strokes are illustrated via dashed lines, which are applied to an image definition 320 to create an effect mask 330 for each stroke. By creating an effect mask 330 rather than pushing a spectral line along a stroke, greater effect consistency across strokes and ink objects may be achieved.” Taylor)
As per claim 17, Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley already teaches “the method according to claim 14”, however Yamamoto in view of Oh, Wang, Hu, Kuwabara, Lebovitz and Hurley does not teach “wherein an object included in the set region is a character.”.
Taylor teaches “wherein an object included in the set region is a character.” (See Fig. 3. It includes a character (310) in the set region (320). Motivated for the already established reasons. Taylor)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DYLAN J MENDEZ MUNIZ whose telephone number is (703)756-5672. The examiner can normally be reached M-F, 8AM - 5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DYLAN JOHN MENDEZ MUNIZ/Examiner, Art Unit 2675
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights