DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is responsive to Applicant’s response received on July 15, 2026. Claims 1-7, 9-11 and 13-23 are pending.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “processing section that generates”, “image inspector that inspects”, and “marking section that applies” in claims 1, 13, 15, 17 and 19.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
The previous 101 rejections have been withdrawn in light of Applicant’s amendment and remarks.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 11, 13, 18, 19 and 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Blohm et al. (US 2004/0265095) and Van Acquoij (US 2016/0103634).
Regarding claim 1, Blohm et al. discloses an inspection system that inspects a bundle including a plurality of processed recording media, the inspection system comprising:
a sensor that acquires information regarding a shape of the bundle including the plurality of processed recording media (“Each book is inspected at the station 18 by a caliper or other sensor to detect the thickness or print quality of the book” at paragraph 0023, line 5); and
a hardware processor, wherein the hardware processor inspects the bundle based on the information acquired by the sensor (“This information is transmitted to the controller 16, which compares the measured thickness or print quality with a reference thickness or print quality, in order to determine if the book has been appropriately assembled” at paragraph 0023, second to last sentence).Blohm et al. does not explicitly disclose that the information acquired by the acquirer is three-dimensional information; and
sends a control command to an ejector, the command including instructions for the ejector to switch a conveyance destination of the bundle in accordance with a result of the inspection by the hardware processor (“If an error was made in the assembly of the book, the book is rejected from the binding line 10 at a divert gate 22” at paragraph 0023, last sentence).
Blohm et al. does not explicitly disclose a sensor that acquires three-dimensional information.
Van Acquoij teaches a system in the same field of endeavor of printed matter defect detection, inspection system comprising:
a sensor that acquires three-dimensional information regarding a shape of the printed matter (“A sensor device 22 in the form of an optical sensor, such as a laser scanner, is provided within the sentry unit 21 for sensing the surface geometry or topology of the sheets S as they travel on a first pass or a second pass along the transport path P. The laser scanner or optical sensor device 22 generates digital image data I of the three-dimensional surface geometry or topology of each sheet S sensed or scanned” at paragraph 0100, line 1); and
a hardware processor, wherein the hardware processor inspects the printed matter based on the three-dimensional information acquired by the sensor (“As the apparatus of the invention employs data representative of the surface geometry or topology of the sheet (i.e. three-dimensional data), the invention is capable of detecting multiple deformation types. Any deformation present within the sheet can be detected by using a full sheet topology measurement, i.e. a 3D image of the sheet” at paragraph 0037, second to last sentence).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize the 3D sensor as taught by Van Acquoij in the system of Blohm et al. to be able to detect a variety of paper deformation types (see Van Acquoij at paragraph 0037).
Regarding claim 22, Blohm et al. discloses an inspection method for inspecting a bundle including a plurality of processed recording media, the inspection method comprising:
acquiring information regarding a shape of the bundle including the plurality of processed recording media (“Each book is inspected at the station 18 by a caliper or other sensor to detect the thickness or print quality of the book” at paragraph 0023, line 5);
inspecting the bundle based on the information acquired by the acquirer (“This information is transmitted to the controller 16, which compares the measured thickness or print quality with a reference thickness or print quality, in order to determine if the book has been appropriately assembled” at paragraph 0023, second to last sentence); and
sending a control command to an ejector, the command including instructions for the ejector to switch a conveyance destination of the bundle in accordance with a result of the inspection by the hardware processor (“If an error was made in the assembly of the book, the book is rejected from the binding line 10 at a divert gate 22” at paragraph 0023, last sentence).
Blohm et al. does not explicitly disclose acquiring three-dimensional information.
Van Acquoij teaches a method in the same field of endeavor of printed matter defect detection, inspection method comprising:
acquiring three-dimensional information regarding a shape of the printed matter (“A sensor device 22 in the form of an optical sensor, such as a laser scanner, is provided within the sentry unit 21 for sensing the surface geometry or topology of the sheets S as they travel on a first pass or a second pass along the transport path P. The laser scanner or optical sensor device 22 generates digital image data I of the three-dimensional surface geometry or topology of each sheet S sensed or scanned” at paragraph 0100, line 1); and
inspecting the printed matter based on the three-dimensional information acquired by the sensor (“As the apparatus of the invention employs data representative of the surface geometry or topology of the sheet (i.e. three-dimensional data), the invention is capable of detecting multiple deformation types. Any deformation present within the sheet can be detected by using a full sheet topology measurement, i.e. a 3D image of the sheet” at paragraph 0037, second to last sentence).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize the 3D sensor as taught by Van Acquoij in the system of Blohm et al. to be able to detect a variety of paper deformation types (see Van Acquoij at paragraph 0037).
Regarding claim 23, Blohm et al. discloses a non-transitory recording medium storing a computer-readable inspection program for inspecting a bundle including a plurality of processed recording media, the inspection program causing a computer to execute:
acquiring information regarding a shape of the bundle including the plurality of processed recording media (“Each book is inspected at the station 18 by a caliper or other sensor to detect the thickness or print quality of the book” at paragraph 0023, line 5);
inspecting the bundle based on the information acquired by the acquirer (“This information is transmitted to the controller 16, which compares the measured thickness or print quality with a reference thickness or print quality, in order to determine if the book has been appropriately assembled” at paragraph 0023, second to last sentence); and
sending a control command to an ejector, the command including instructions for the ejector to switch a conveyance destination of the bundle in accordance with a result of the inspection by the hardware processor (“If an error was made in the assembly of the book, the book is rejected from the binding line 10 at a divert gate 22” at paragraph 0023, last sentence).
Blohm et al. does not explicitly disclose acquiring three-dimensional information.
Van Acquoij teaches a method in the same field of endeavor of printed matter defect detection, inspection method comprising:
acquiring three-dimensional information regarding a shape of the printed matter (“A sensor device 22 in the form of an optical sensor, such as a laser scanner, is provided within the sentry unit 21 for sensing the surface geometry or topology of the sheets S as they travel on a first pass or a second pass along the transport path P. The laser scanner or optical sensor device 22 generates digital image data I of the three-dimensional surface geometry or topology of each sheet S sensed or scanned” at paragraph 0100, line 1); and
inspecting the printed matter based on the three-dimensional information acquired by the sensor (“As the apparatus of the invention employs data representative of the surface geometry or topology of the sheet (i.e. three-dimensional data), the invention is capable of detecting multiple deformation types. Any deformation present within the sheet can be detected by using a full sheet topology measurement, i.e. a 3D image of the sheet” at paragraph 0037, second to last sentence).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize the 3D sensor as taught by Van Acquoij in the system of Blohm et al. to be able to detect a variety of paper deformation types (see Van Acquoij at paragraph 0037).
Regarding claim 11, Van Acquoij discloses a system wherein the three-dimensional information is represented by a two-dimensional heat map or two-dimensional contour lines (“In a first processing step, a binary image is created where every pixel exceeding the preset height threshold given by TOL is set to 1, all other pixels are set to 0. The minimum threshold level for detecting defects is preferably set to 400 μm, as it has been found that a lower level will result in detection of too many very small defects. The processor device 24 produces a height map for each sheet. This height map is used to detect and measure or classify any defects present within the sheet, and particularly any out-of-plane deformations D, such as wrinkles, dog ears, curl, tears etc. In this embodiment, a defect is defined as a measurement point within the height map having at least 4 connected neighbors also exceeding a preset threshold value” at paragraph 0102).
Regarding claim 13, Blohm et al. discloses system further comprising a processing section that generates a bundle including a plurality of recording media by processing, wherein the hardware processor inspects the bundle processed by the processing section (“The binding line 10 includes a controller 16, as is known in the art, that maintains a master mailing list having a prearranged sequence or order, such as, for example, a zip code order. The controller 16 controls the feeders 12 to assemble each book according to the mailing list information. The controller 16 monitors the position of each book as it is being assembled on the conveyor 14 and tracks the position of each book as it moves downstream of the feeders 12” at paragraph 0022).
Regarding claim 18, Blohm et al. discloses a system comprising the ejector that switches the conveyance destination of the bundle in accordance with the result of the inspection by the hardware processor (“If an error was made in the assembly of the book, the book is rejected from the binding line 10 at a divert gate 22” at paragraph 0023, last sentence).
Regarding claim 19, Blohm et al. discloses a system further comprising a marking section that applies marking to the bundle in accordance with a result of the inspection by the hardware processor (“Each acceptable book may be further personalized by printers 30 and 31, as illustrated at step 100” at paragraph 0032, second to last sentence).
Regarding claim 21, Blohm et al. discloses a system wherein, when the hardware processor determines that the bundle to be inspected is a defective product, the hardware processor records information regarding the defective product (“The recipients of the rejected books are identified in the mailing list and stored in the controller” at paragraph 0033, line 1).
Claim(s) 2-4, 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Blohm et al. and Van Acquoij as applied to claim 1 above, and further in view of Zamir et al. (US 2023/0153979).
Regarding claim 2, the Blohm et al. and Van Acquoij combination discloses a system as described in claim 1 above.
The Blohm et al. and Van Acquoij combination does not explicitly disclose that the hardware processor inspects the bundle using a deep learning model.
Zamir et al. teaches a system in the same field of endeavor of printed matter inspection, wherein the hardware processor inspects the printing using a deep learning model (“The image-based feature extractor may include unsupervised (e.g. unlabeled) training where the input and output are the same image, thereby obtaining a deep encoding network for extracting the image features” at paragraph 0039, second to last sentence).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize a deep network as taught by Zamir et al. in the bundle defect evaluation of the Blohm et al. and Van Acquoij combination as “usage during online inspection to rule-out ‘pure’ false-alarms (e.g., unambiguously attributed to differences unassociated with the actual defective material such as illumination and sensor noise). This allows the system to safely set classic-machine-vision-algorithms based inspection thresholds to ‘over-report’, thereby addressing issues of misdetection as well” (Zamir et al. at paragraph 0043, third to last sentence).
Regarding claim 3, Zamir et al. discloses a system wherein the hardware processor uses a model of an unsupervised learning method as the deep learning model (“The image-based feature extractor may include unsupervised (e.g. unlabeled) training where the input and output are the same image, thereby obtaining a deep encoding network for extracting the image features” at paragraph 0039, second to last sentence).
Regarding claim 4, Zamir et al. discloses a system wherein the hardware processor uses an autoencoder as the deep learning model (“As described above, bandwidth may be reduced by applying the feature-extraction block (e.g. based on convolutional-layers auto-encoder framework) on the edge (e.g. where the image data is being saved during online inspection), thus obtaining a compressed feature vector best preserving salient-for-detection/classification information that is passed to a server-entity to be concatenated with additional ‘contextual’ features for the auto-editing phase” at paragraph 0043, line 1).
Regarding claim 10, Van Acquoij discloses a system, wherein
the hardware processor generates an image of the three-dimensional information (“In a first processing step, a binary image is created where every pixel exceeding the preset height threshold given by TOL is set to 1, all other pixels are set to 0. The minimum threshold level for detecting defects is preferably set to 400 μm, as it has been found that a lower level will result in detection of too many very small defects. The processor device 24 produces a height map for each sheet. This height map is used to detect and measure or classify any defects present within the sheet, and particularly any out-of-plane deformations D, such as wrinkles, dog ears, curl, tears etc. In this embodiment, a defect is defined as a measurement point within the height map having at least 4 connected neighbors also exceeding a preset threshold value” at paragraph 0102), and
data of the image generated by the hardware processor is used for the learning model (“the processing device might contain an auto-learning algorithm for improving its classification abilities during continued operation” at paragraph 0020, line 9).
Regarding claim 20, the Blohm et al. and Van Acquoij combination discloses a system further comprising a system controller (controller 16 as described above is a system controller), wherein the processing includes stapling (“Downstream of the station 18, the binding line 10 includes a stitcher 24, where the books are bound, i.e., stapled, glued, stitched, and fastened” Blohm et al. at paragraph 0024, line 1).
The Blohm et al. and Van Acquoij combination does not explicitly disclose that the system controller stops operation of the system when the hardware processor determines a defect related to the stapling.
Zamir et al. teaches a system in the same field of endeavor of printed matter inspection, comprising a system controller wherein the system controller stops operation of the system when the hardware processor determines a defect (“In exemplary methods, the step of accepting at least some defect alerts (automatically or via the user interface) may prompt providing a feedback signal to a controller for the production line effective to stop the production line, to take an action on the defective product instances, or to otherwise impact the behavior of the production line” at paragraph 0047, line 4).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize the machine vision system as taught by Zamir et al. in the bundle defect evaluation of the Blohm et al. and Van Acquoij combination as “usage during online inspection to rule-out ‘pure’ false-alarms (e.g., unambiguously attributed to differences unassociated with the actual defective material such as illumination and sensor noise). This allows the system to safely set classic-machine-vision-algorithms based inspection thresholds to ‘over-report’, thereby addressing issues of misdetection as well” (Zamir et al. at paragraph 0043, third to last sentence).
Claim(s) 16 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Blohm et al. and Van Acquoij as applied to claim 1 above, and further in view of Nishi (US 2022/0334520).
The Blohm et al. and Van Acquoij combination discloses a system as described in claim 1 above.
The Blohm et al. and Van Acquoij combination does not explicitly disclose that the acquirer acquires the information using light emitted to the bundle.
Nishi teaches a system in the same field of endeavor of printed matter defect detection, wherein the acquirer acquires the information using light emitted to the bundle (“The basis weight detector is a sensor that detects the basis weight of the paper sheet 90, includes a light emitter and a light receiver, and measures the basis weight by an attenuation amount of light transmitted through the paper sheet 90. For example, a basis weight sensor has the light emitter disposed below the paper sheet passing area (paper sheet passing path) into which the paper sheet is inserted, and the light receiver disposed above the paper sheet passing area. In this configuration, the paper sheet 90 passes between the light emitter and the light receiver, and the basis weight sensor detects the basis weight of the paper sheet 90 based on intensity of the light received by the light receiver to output the measurement result (measurement value 2)” at paragraph 0068).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize the paper discrimination as taught by Nishi in the inspection model of the Blohm et al. and Van Acquoij combination to enable the system to determine when the incorrect paper type is utilized in the printing.
Claim(s) 14 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Blohm et al. and Van Acquoij as applied to claim 13 above, and further in view of Manabe (US 2022/0201150).
Regarding claim 14, the Blohm et al. and Van Acquoij combination discloses a system as described in claim 1 above.
The Blohm et al. and Van Acquoij combination does not explicitly disclose a printer that prints images on the recording media, wherein the plurality of recording media on which the images have been printed by the printer are processed by the processing section.
Manabe teaches a system in the same field of endeavor of printed matter defect detection comprising a printer that prints images on the recording media (“The cut printed sheets are formed into sheet bundles in units of books by the buffer unit 0115. In the present embodiment, the buffer unit 0115 is connected in-line to a continuous feed printer postprocessor 0116 by a conveyance path. The image formed (printed) sheet bundles (paper bundles) in units of books are sent to the continuous feed printer postprocessor 0116, and are supplied as a target of processing by a continuous feed printer postprocessor 0116” at paragraph 0042, line 10), wherein the plurality of recording media on which the images have been printed by the printer are processed by the processing section (“The number of reference images to be read is the number of pages to he inspected (front side and back side of each sheet). In the inspection unit 0106, the conveyance of the sheet is detected by a sensor, for example, and the reference image is acquired by scanning the detected sheet” at paragraph 0084, line 1).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize the system of the Blohm et al. and Van Acquoij combination in direct conjunction with a printing workflow as taught by Manabe as “even if a defect in the print materials is found by the inspection apparatus 0108, it is converted into an inspection result (in units of books) and transmitted to the postprocessor control apparatus 0118, so that the image forming apparatus can continue printing without performing reprinting. Therefore, it is not necessary to prepare for reprinting or perform reprinting processing itself, and the productivity of the cutsheet device can be improved” (Manabe at paragraph 0146, line 2).
Regarding claim 15, Manabe discloses a system comprising an image inspector that inspects the images printed on the recording media by the printer, wherein the plurality of recording media on which the images have been inspected by the image inspector are processed by the processing section (“The image forming apparatus 0101 further includes an inspection unit 0106 and a large-capacity stacker 0107. The inspection unit 0106 is connected to the inspection apparatus 0108 via a cable 0114. A sheet on which an image has been formed by the image forming apparatus 0101 is read by the inspection unit 0106, and the quality of an image turned on the sheet is inspected by the inspection apparatus 0108” at paragraph 0041, line 1).
Claim(s) 17 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Blohm et al. and Van Acquoij as applied to claim 1 above, and further in view of Nakano (US 2021/0374939).
The Blohm et al. and Van Acquoij combination discloses a system as described in claim 1 above.
The Blohm et al. and Van Acquoij combination does not explicitly disclose that the sensor includes a first sensor that acquires information regarding a shape of the bundle on a front side of the bundle, and a second sensor that acquires information regarding a shape of the bundle on a back side of the bundle.
Nakano teaches a system in the same field of endeavor of printed matter defect detection wherein the sensor includes a first sensor that acquires information regarding a shape of the bundle on a front side of the bundle, and a second sensor that acquires information regarding a shape of the bundle on a back side of the bundle (“The lower scanner 521 reads a lower surface of a printed matter (the sheet on which the printed image is formed by the image forming device 300) conveyed along the conveyance path. That is, the lower scanner 521 reads the printed image (formed on a front surface) of the printed matter that is back printed from a back surface side of the sheet through the sheet of the printed matter. The upper scanner 522 reads an upper surface of the printed matter conveyed along the conveyance path. That is, the upper scanner 522 reads the printed image of the printed matter that is back printed from a print surface side (front surface side) of the printed matter” at paragraph 0052, line 1).
It would have been obvious to one ordinary skill in the art before the effective filing date of the invention to utilize the dual imaging as taught by Nakano on the bundle of the Blohm et al. and Van Acquoij combination to ensure that the system is able to detect the visual condition of both sides of the bundle.
Claim(s) 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Blohm et al., Van Acquoij and Zamir et al. further in view of Nishi (US 2022/0334520).
Regarding claim 5, the Blohm et al., Van Acquoij and Zamir et al. combination discloses a system as described in claim 2 above.
The Blohm et al., Van Acquoij and Zamir et al. combination does not explicitly disclose that the deep learning model is a model trained for each basis weight of a cover sheet of the bundle, and when the hardware processor inspects the bundle using the deep learning model, the hardware processor refers to a model corresponding to the basis weight of the cover sheet of the bundle.
Nishi teaches a system in the same field of endeavor of printed matter defect detection, wherein
the learning model is a model trained for each basis weight of a cover sheet (“Furthermore, the image forming apparatus 10 performs paper type discrimination from the measurement result (measurement values 1 to 3) obtained from the medium detection apparatus 40. The storage 12 of the image forming apparatus 10 stores a learning model obtained by machine learning (also referred to as a trained model) used for paper type discrimination. This is a learning model generated by supervised learning using training data, with detection output of the medium detection apparatus 40 for the paper sheet 90 as an input value and paper type information of the paper sheet 90 set by the user as a correct label. The controller 11 of the image forming apparatus 10 obtains a basis weight conversion value, a paper thickness conversion value, and a surface property measurement value by using the measurement result (measurement values 1 to 3) obtained by measuring the paper sheet 90 by using the paper thickness detector, the basis weight detector, and the surface property detector of the medium detection apparatus 40” at paragraph 0072, line 1), and
when the hardware processor inspects the printed matter using the learning model, the hardware processor refers to a model corresponding to the basis weight of the cover sheet (“Furthermore, the paper sheet information may be associated with the paper sheet feeding tray that uses a paper sheet physical property measured by the medium detection apparatus 40 and/or a paper type (paper sheet type) determined from the paper sheet physical property, and a basis weight, and this may be used as the paper information” at paragraph 0082, last sentence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the paper discrimination as taught by Nishi in the inspection model of the Blohm et al., Van Acquoij and Zamir et al. combination to enable the system to determine when the incorrect paper type is utilized in the printing.
Regarding claim 6, the Blohm et al., Van Acquoij and Zamir et al. combination discloses a system as described in claim 1 above.
The Blohm et al., Van Acquoij and Zamir et al. combination does not explicitly disclose that the deep learning model is a model trained for each number of recording media forming the bundle, and when the hardware processor inspects the bundle using the deep learning model, the hardware processor refers to a model corresponding to the number of recording media forming the bundle.
Nishi teaches a system in the same field of endeavor of printed matter defect detection, wherein
the learning model is a model trained for the thickness of recording media (“Furthermore, the image forming apparatus 10 performs paper type discrimination from the measurement result (measurement values 1 to 3) obtained from the medium detection apparatus 40. The storage 12 of the image forming apparatus 10 stores a learning model obtained by machine learning (also referred to as a trained model) used for paper type discrimination. This is a learning model generated by supervised learning using training data, with detection output of the medium detection apparatus 40 for the paper sheet 90 as an input value and paper type information of the paper sheet 90 set by the user as a correct label. The controller 11 of the image forming apparatus 10 obtains a basis weight conversion value, a paper thickness conversion value, and a surface property measurement value by using the measurement result (measurement values 1 to 3) obtained by measuring the paper sheet 90 by using the paper thickness detector, the basis weight detector, and the surface property detector of the medium detection apparatus 40” at paragraph 0072, line 1),
when the hardware processor inspects the printed matter using the learning model, the hardware processor refers to a model corresponding to the thickness of recording media (“Furthermore, the paper sheet information may be associated with the paper sheet feeding tray that uses a paper sheet physical property measured by the medium detection apparatus 40 and/or a paper type (paper sheet type) determined from the paper sheet physical property, and a basis weight, and this may be used as the paper information” at paragraph 0082, last sentence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the paper discrimination as taught by Nishi in the inspection model of the Blohm et al., Van Acquoij and Zamir et al. combination to enable the system to determine when the incorrect paper type is utilized in the printing.
Therefore, the combined teachings of paper thickness modeling in addition to the disclosure of Blohm et al. evaluating the imaged bundle for the expected overall thickness, one may therefore derive the number of pages for the bundle given these two values.
Regarding claim 7, the Blohm et al., Van Acquoij and Zamir et al. combination discloses a system as described in claim 1 above.
The Blohm et al., Van Acquoij and Zamir et al. combination does not explicitly disclose that the deep learning model is a model trained for each combination of the basis weight of the cover sheet of the bundle and the number of recording media forming the bundle, and when the hardware processor inspects the bundle using the deep learning model, the hardware processor refers to a model corresponding to the combination.
Nishi teaches a system in the same field of endeavor of printed matter defect detection, wherein
the learning model is a model trained for each combination of the basis weight of the cover sheet of the bundle and the thickness of recording media (“Furthermore, the image forming apparatus 10 performs paper type discrimination from the measurement result (measurement values 1 to 3) obtained from the medium detection apparatus 40. The storage 12 of the image forming apparatus 10 stores a learning model obtained by machine learning (also referred to as a trained model) used for paper type discrimination. This is a learning model generated by supervised learning using training data, with detection output of the medium detection apparatus 40 for the paper sheet 90 as an input value and paper type information of the paper sheet 90 set by the user as a correct label. The controller 11 of the image forming apparatus 10 obtains a basis weight conversion value, a paper thickness conversion value, and a surface property measurement value by using the measurement result (measurement values 1 to 3) obtained by measuring the paper sheet 90 by using the paper thickness detector, the basis weight detector, and the surface property detector of the medium detection apparatus 40” at paragraph 0072, line 1), and
when the hardware processor inspects the printed using the learning model, the hardware processor refers to a model corresponding to the combination (“Furthermore, the paper sheet information may be associated with the paper sheet feeding tray that uses a paper sheet physical property measured by the medium detection apparatus 40 and/or a paper type (paper sheet type) determined from the paper sheet physical property, and a basis weight, and this may be used as the paper information” at paragraph 0082, last sentence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the paper discrimination as taught by Nishi in the inspection model of the Blohm et al., Van Acquoij and Zamir et al. combination to enable the system to determine when the incorrect paper type is utilized in the printing.
Therefore, the combined teachings of paper thickness modeling in addition to the disclosure of Blohm et al. evaluating the imaged bundle for the expected overall thickness, one may therefore derive the number of pages for the bundle given these two values.
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Blohm et al., Van Acquoij and Zamir et al. as applied to claim 2 above, and further in view of d'Armancourt et al. (US 10,834,283).
The Blohm et al., Van Acquoij and Zamir et al. combination discloses a system as described in claim 2 above.
The Blohm et al., Van Acquoij and Zamir et al. combination does not explicitly disclose that the hardware processor normalizes the three-dimensional information according to a thickness and a size of the bundle, and the deep learning model uses data normalized by the hardware processor.
d'Armancourt et al. teaches a system in the same field of endeavor of printed matter defect detection, wherein
the hardware processor normalizes the three-dimensional information according to a thickness and a size of the bundle (“The image may be captured by any method of digital image capture, such as imagers, image sensors (such as an integrated 1D, 2D, or 3D image sensor)” at col. 11, line 9; “FIG. 4 shows a method 400 for image processing, according to an embodiment. At step 402, a size of a produced image is normalized to match a size of a reference image” at col. 15, line 3; “Normalizing a size of a produced image at step 402 can include adjusting a size of the produced image to align corners and/or edges of the produced image with corners and/or edges of the reference image. Additionally or alternatively, normalizing a size of a produced image at step 402 can include equalizing a distance between a rightmost printed area and a leftmost printed area of the image” at col. 15, line 25; though not explicit, the reference image has a particular size associated with the object of interest and in the case of Blohm et al., the expected thickness of the book is to be examined), and
the learning model uses data normalized by the hardware processor (“And at step 406, a difference image is produced by comparing the normalized produced image to the reference image” at col. 15, line 7; “The database can be self-learning, and take into account the frequency of occurrence of each detected issue, to improve determination of the root cause and/or issue resolution in the future” at col. 29, line 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the normalization as taught by d'Armancourt et al. in the system of the Blohm et al., Van Acquoij and Zamir et al. combination to ensure the image is processed according to a standardized size condition for fair evaluation of whether there is a defect present or not.
Response to Arguments
Summary of Remarks (@ response page labeled 8): “With exemplary reference to paragraph [0071] of the subject application, the ejector receives the control command from the hardware processor, and switches the conveyance destination of the bundle from the non-defective product collecting section to the defective product collecting section. The inspection system can continue production of booklets or the like without stopping the operation of the system even in a case where the system receives the defective bundle. This provides a specific improvement over prior systems.”
Examiner’s Response: This argument is found to persuasive. As such, the 101 rejections have been withdrawn as noted above.
Summary of Remarks (@ response page labeled 9): “However, it is respectfully submitted that Van Acquoij fails to teach or suggest a sensor that acquires three-dimensional information regarding a shape of a bundle including a plurality of processed recording media. Instead, Van Acquoij teaches a sensor device for sensing a surface geometry or topology of a sheet to be printed and for generating data that is representative of that surface geometry or topology of the sheet. See paragraphs [0035] and [0037]. Van Acquoij's sensor merely senses a surface geometry or topology of a single sheet to be printed, and does not sense a shape of a bundle including a plurality of processed recording media.”
Examiner’s Response: While it is recognized the Van Acquoij does not specifically disclose imaging a stack of paper, it is asserted that the inspection principles afforded by the three-dimensional processing of Van Acquoij would apply to inspection of a stack of papers. In particular, as Applicant points out that Van Acquoij generates inspection information using the topology of the printed matter, a surface defect of a single sheet is translatable to a surface defect of a stack of paper. In both cases, the variable height caused by the defect is able to be detected. As such, together with Blohm et al., one of ordinary skill in the art would be able to image the topology of a paper stack and detect any surface defects.
Summary of Remarks (@ response page labeled 9): “Moreover, the combination of Blohm and Van Acquoij still fails to teach or suggest a hardware processor that "inspects the bundle based on the three-dimensional information acquired by the sensor, and sends a control command to an ejector, the control command including instructions for the ejector to switch a conveyance destination of the bundle in accordance with a result of the inspection by the hardware processor." Specifically, Blohm teaches a system including a sensor that detects the thickness or print quality of the book, and a controller that compares the measured thickness or print quality with a reference thickness or print quality, in order to determine if the book has been appropriately assembled. See Blohm, paragraph [0023]. Blohm's controller does not inspect the book based on three-dimensional information.”
Examiner’s Response: Blohm et al. discloses a mechanism to reject and divert defective printed stacks upon determination that the inspection flagged a problem. While Blohm et al. does not state that the result is dependent on three-dimensional information, Van Acquoij utilizes 3D image information to generate a height map that indicates the presence of any defects. Therefore, the combination as asserted above discloses the limitations.
Summary of Remarks (@ response page labeled 10): “Furthermore, there is no motivation to modify the system of Blohm by the sensor taught by Van Acquoij. Blohm's sensor detects the thickness or print quality of the book in order to determine if the book has been appropriately assembled. In contrast, Van Acquoij's sensor senses a surface geometry or topology of a single sheet to be printed. Van Acquioij's sensor is located on a conveyance path opposite Blohm's sensor with respect to a printer.”
Examiner’s Response: Applicant notes that Blohm et al. observes the thickness of the book. Van Acquioij observes the topology, or measures of variable thickness, in the imaged printed matter. As both references deal with print quality and observation of printed matter thickness, there is a reasonable expectation of combination. The reasons to combine the two is outlined in the rejections above, as Van Acquioij demonstrates how measuring the variations in thickness can signal specific types of printed defects. These would be beneficial to flag before final binding (see Van Acquioij at paragraph 0026). As to the location of the sensors, it was not asserted that the specific physical configuration within the inspection system of Van Acquioij’s sensor would modify Blohm’s configuration, only that it would be obvious to use that type of sensor in the inspection instead of the two-dimensional sensor of Blohm et al.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Slocumb et al. (US 20230084769) is pertinent as disclosing a print quality inspection system that utilizes a 3D scanner to observe the shape and thickness of the inspection item.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 9:30-5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/Primary Examiner, Art Unit 2672
Prosecution Insights