DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page 9, filed 6/25/26, with respect to claim 17 have been fully considered and are persuasive. The objection has been withdrawn.
Applicant's arguments filed 6/25/26 have been fully considered but they are not persuasive.
Regarding applicant’s argument for claim 1, on page 9, that Kondo does not disclose identifying a typology of an error because one of ordinary skill would not recognize a pixel value as a tag, examiner disagrees. Kondo establishes three main methods for introducing error into the generated training data (paragraph 102), each one creating noisy image data (figure 4, item S185) that is then used as ground truth data (figure 10a, item S320) for iterating during training. Comparing an output to a ground truth (i.e. the pixel values) is exactly the function of a “tag” in machine learning and would be recognized as such by one of ordinary skill. Examiner notes that simply because Kondo discloses a very broad category of image (i.e. standard vs. nonstandard) does not preclude more precise categorization within the group of “nonstandard”. Examiner further notes that applicant themselves appear to be utilizing pixel values as vectors to denote “an extension” of the error as claimed in claim 6, thereby clearly suggesting pixels values used as tag data. Therefore the argument is overcome and the previous rejection remains.
Applicant’s arguments, see page 10, filed 6/25/26, with respect to the rejection(s) of claim(s) 20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of U.S. patent application publication 2019/0248153 by Muehl et al.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
1) Claim(s) 1-9, 11, 12, 14, 16-18, 21 and 22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. patent application publication 2021/0097656 by Kondo.
2) Regarding claim 1, Kondo teaches a computer implemented method for automatic generation of training images for a machine learning system for image recognition, the machine learning system being usable by a digital printer, wherein the method comprises: obtaining, from at least one database of images, a series of initial digital images (figure 4, item S105; paragraph 59; initial images can be obtained through scanning sheets, image storage within MFP being analogous to a database); applying to each of the initial digital images a first printing error representative of a given malfunction of the digital printer when printing a digital image on physical material (paragraphs 51 and 60; “nonstandard” pattern [i.e. a defective pattern] can be selected to apply to the sheet image); automatically tagging each of the initial digital images with the first printing error to obtain a series of tagged images (paragraphs 71-76; figure 6A; base pattern is generated for each pixel in a line by line manner, the setting of pixel values being “tagging”), wherein tagging comprises identifying a typology of the first printing error (Kondo establishes three main methods for introducing error into the generated training data [paragraph 102], each one creating noisy image data [figure 4, item S185] that is then used as ground truth data [figure 10a, item S320] for iterating during training, comparing an output to a ground truth [i.e. the pixel values] is exactly the function of a “tag” in machine learning and would be recognized as such by one of ordinary skill); graphically manipulating each of the tagged initial digital images to obtain, from each of the tagged images, one or more rendered images, wherein each respective one or more rendered images has appearance of a respective sheet material for printing (paragraphs 78, 83 and 86; background image is generated to reproduce an image of the sheet itself and is combined with base pattern data to generate “rendered” pattern data detailed in S175 of figure 4); and providing the obtained rendered images for use in training the machine learning system to detect potential printing errors to signal malfunction in the digital printer (figure 10A; paragraph 105-107; produced noisy pattern image data is utilized to train a learning model).
3) Regarding claim 2, Kondo teaches the method of claim 1, wherein applying to each of the initial digital images the first printing error comprises: identifying a selection area within each initial digital image where to apply the first printing error; and altering at least one graphic property of each respective selection area (paragraph 70; columns for error addition are randomly selected).
4) Regarding claim 3, Kondo teaches the method of claim 2, wherein identifying the selection area where to apply the first printing error comprises: identifying a primary pixel in each of the initial digital images; and selecting one or more secondary pixels correlated to each other and to the primary pixel (paragraph 67; a first pixel is selected and then pixels are altered in a line).
5) Regarding claim 4, Kondo teaches the method of claim 3, wherein selecting the one or more secondary pixels comprises selecting one or more pixels aligned to the primary pixel parallel to a direction of extension of the selection area (paragraphs 67 and 70; nonstandard lines are built pixel by pixel extending in a rectangle of random width).
6) Regarding claim 5, Kondo teaches the method of claim 2, wherein altering the at least one graphic property of the selection area comprises: analyzing a color tone of the primary and secondary pixels of the selection area; and modifying the color tone of one or more of the primary and secondary pixels whose color tone has been analyzed by inserting pixels having color tone different from the color tone of the pixels of a respective initial digital image (paragraph 71; color luminance values are set for nonstandard lines different from the standard lines).
7) Regarding claim 6, Kondo teaches the method of claim 1, wherein tagging with the first printing error comprises identifying at least one of: a position with respect to a reference system and an extension of the first printing error (paragraph 70; the typology is the widths and positions of the nonstandard lines tagged according to the values of the altered pixels which is both a position with respect to a reference [e.g. a nonstandard line with a nonstandard width] and an “extension” [i.e. a nonstandard line extends in a particular direction]).
8) Regarding claim 7, Kondo teaches the method of claim 1, wherein each of the rendered images includes a surface aspect corresponding to a real support where the initial digital images are to be printed (paragraphs 59 and 83; sheet image corresponds to actual surface of media).
9) Regarding claim 8, Kondo teaches the method of claim 1, wherein graphically manipulating each of the tagged images comprises applying to each of the tagged images one or more of the following maps: an optical map defining a series of optical properties, a structural map defining a series of structural properties, a material map defining a series of properties related to the material of a support on which the printing may be performed, and a light map defining a series of illumination properties; and wherein each of the optical map, the structural map, the material map, the light map is a filter configured for treating a sub-region of each tagged image (paragraph 71; luminance values for nonstandard [i.e. defective] line [a line being a sub-region] is applied to pixel data).
10) Regarding claim 9, Kondo teaches the method of claim 8, wherein graphically manipulating each of the tagged images comprises applying, to each of the tagged images, the optical map defining the series of optical properties, and wherein the series of optical properties comprises at least one of reflection, diffusion, transparency, and distortion (paragraph 71 and 102; random width data adds distortions).
11) Regarding claim 11, Kondo teaches the method of claim 8, wherein graphically manipulating each of the tagged images comprises applying, to each of the tagged images, the material map defining the series of properties related to the material of the support on which the printing may be performed, wherein the series of properties related to the material of the support on which the printing may be performed comprises at least one of textile material, leather, paper, and wood (paragraphs 46 and 59; at least fabric [i.e. a textile] and different types of paper are disclosed as generated background data [figure 4, item S155]).
12) Regarding claim 12, Kondo teaches the method of claim 8, wherein graphically manipulating each of the tagged images comprises applying, to each of the tagged images, the light map defining a series of illumination properties, wherein the series of illumination properties comprises at least one of shading and luminosity (paragraph 71; luminance values for nonstandard [i.e. defective] line [a line being a sub-region] is applied to pixel data).
13) Regarding claim 14, Kondo teaches the method of claim 8, wherein the graphical manipulation of each of the tagged images is performed iteratively for each property of each series of properties of a respective map of the maps, wherein the method comprises: varying one or more properties of each series of properties of a respective map to generate from the same tagged image, a further rendered image, wherein the further rendered images, obtained by varying the properties of each series of properties of the respective map applied to tagged images, are different to each other (figure 4; process is performed iteratively on the same scanned image data as shown with random elements applied to produce the nonstandard lines [paragraph 70]).
14) Regarding claim 16, Kondo teaches the method of claim 1, wherein obtaining the series of initial digital images comprises generating or collecting from the at least one database of images, a plurality of series of initial digital images, wherein each series of the plurality of series of initial digital images is obtained by dividing an initial macro-image in units which define a respective initial image (paragraph 53; images can be cropped).
15) Regarding claim 17, Kondo teaches the method of claim 1, comprising: training the machine learning system using the one or more rendered images (figure 10A; model is trained).
16) Regarding claim 18, Kondo teaches the method of claim 17, comprising: receiving one or more signals generated by an optical sensor of the digital printer; determining one or more image samples of a printed image as a function of the one or more signals received by the optical sensor, and determining, using the trained machine learning system, an alarm condition depending on the one or more image samples by performing a comparison of the one or more image samples with the one or more rendered images (figure 12; paragraph 128; pattern is printed and scanned and the learned model is applied to determine defective nozzles [defective being an “alarm condition”]).
17) Claim 21 is taught in the same manner as described in the rejection of claim 1 above.
18) Claim 22 is taught in the same manner as described in the rejections of claims 1 and 6 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
18) Claim(s) 10, 13 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. patent application publication 2021/0097656 by Kondo as applied to claims 1 and 8 above, and further in view of U.S. patent application publication 2020/0356328 by Christy et al.
19) Regarding claim 10, Kondo does not specifically teach the method of claim 8, wherein graphically manipulating each of the tagged images comprises applying, to each of the tagged images, the structural map defining the series of structural properties, wherein the series of structural properties comprises at least one of thickness, roughness, and porosity of the material.
Christy teaches the method of claim 8, wherein graphically manipulating each of the tagged images comprises applying, to each of the tagged images, the structural map defining the series of structural properties, wherein the series of structural properties comprises at least one of thickness, roughness, and porosity of the material (paragraph 58; porosity is determined and accounted for by a learning model).
Kondo and Christy are combinable because they are both from the printing error machine learning field of endeavor.
It would have been obvious to a person of ordinary skill in the art at the time the invention was effectively filed to combine Kondo with Christy to add porosity of a material as a learning variable. The motivation for doing so would have been “to achieve a desired level of ink saturation” (paragraph 58). Therefore it would have been obvious to combine Kondo with Christy to obtain the invention of claim 10.
20) Regarding claim 13, Kondo (as combined with Christy in the rejection of claim 10 above) teaches the method of claim 8, wherein graphically manipulating each of said tagged images comprises sequentially applying the optical map, the structural map (as taught by Christy), the material map, the light map on adjacent regions, of the tagged image, until complete coverage of the tagged image (figure 6A, items s240 and S245; data is synthetically generated iteratively in units of lines).
21) Regarding claim 15, Kondo (as combined with Christy in the rejection of claim 10 above) teaches the method of claim 1, comprising: applying to each of the initial digital images or to each of the tagged image a different known error; automatically tagging the respective different error and obtaining a series of further tagged images (paragraphs 46 and 148; plurality of errors possible [each differing nonstandard width is technically a different error] and can be labeled), graphically manipulating each of the further tagged images to obtain a plurality of further rendered images, each of which having a surface aspect corresponding to a real support where the initial digital images shall be printed, wherein graphically manipulating each of the further tagged images comprises applying to each of the further tagged images one or more of the following maps: an optical map defining a series of optical properties, a structural map defining a series of structural properties, a material map defining a series of properties related to the material of the support on which the printing may be performed, and a light map defining a series of illumination properties (paragraph 71; luminance values for nonstandard [i.e. defective] line [a line being a sub-region] is applied to pixel data); wherein each of the optical map, the structural map (as taught by Christy), the material map, the light map is a filter configured for treating a sub-region of each further tagged image (figure 6A, items s240 and S245; data is synthetically generated iteratively in sub-regions of lines).
22) Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. patent application publication 2021/0097656 by Kondo as applied to claim 18 above, and further in view of U.S. patent application publication 2019/0210387 by Otsuka et al.
Kondo teaches the method of claim 18, comprising: in response to determining the alarm condition, identifying a defective nozzle of printheads of the digital printer based on the alarm condition (figure 12; paragraph 128; pattern is printed and scanned and the learned model is applied to determine defective nozzles [defective being an “alarm condition”]).
Kondo does not specifically teach sending an instruction to deactivate the defective nozzle and/or to emit an alarm signal.
Otsuka teaches sending an instruction to deactivate the defective nozzle and/or to emit an alarm signal (paragraphs 54 and 67; user can be warned of a defective nozzle).
Kondo and Otsuka are combinable because they are both from the printing error machine learning field of endeavor.
It would have been obvious to a person of ordinary skill in the art at the time the invention was effectively filed to combine Kondo with Otsuka to add notification. The motivation for doing so would have been to warn an operator of an error (paragraph 67). Therefore it would have been obvious to combine Kondo with Otsuka to obtain the invention of claim 19.
23) Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. patent application publication 2021/0097656 by Kondo, and further in view of U.S. patent application publication 2019/0248153 by Muehl et al.
Kondo teaches a digital printer comprising: a printing station having one or more printheads configured to eject ink on sheet material (figure 1, an MFP); an optical sensor located downstream the one or more printheads with respect to a direction of movement of the sheet material, the optical sensor being configured to generate one or more signals representative of an image printed on the sheet material; a control unit connected to the optical sensor, wherein the control unit comprises a trained machine learning system installed thereon, wherein the trained machine learning system is configured to: receiving the one or more signals generated by the optical sensor of the digital printer; determining one or more image samples of the printed image as a function of the one or more signals received by the optical sensor; determining an alarm condition depending on the one or more image samples by performing a comparison of the one or more image samples with one or more rendered images; in response to determining the alarm condition, identifying a defective nozzle of printheads of the digital printer based on the alarm condition (figure 12; printed output is scanned and defective nozzle can be determined according to the learned model).
Kondo does not specifically teach sending an instruction to deactivate the defective nozzle.
Muehl teaches sending an instruction to deactivate the defective nozzle (paragraph 20; defective nozzle can be identified and turned off).
Kondo and Muehl are combinable because they are both from the printing error machine learning field of endeavor.
It would have been obvious to a person of ordinary skill in the art at the time the invention was effectively filed to combine Kondo with Muehl to add nozzle deactivation. The motivation for doing so would have been to compensate for a defective nozzle (paragraph 20). Therefore it would have been obvious to combine Kondo with Muehl to obtain the invention of claim 20.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN O DULANEY whose telephone number is (571)272-2874. The examiner can normally be reached Mon-Fri 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at (571)270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
BENJAMIN O. DULANEY
Primary Examiner
Art Unit 2676
/BENJAMIN O DULANEY/ Primary Examiner, Art Unit 2683
Prosecution Insights