DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Species II (claims 5–6 and 13–14) in the reply filed on August 17th, 2026 is acknowledged. The application has pending claims 1–15 (claims 2 and 7–12 are withdrawn from further consideration).
Specification
The disclosure is objected to because of the following informalities:
In paragraph [0044], the final sentence refers to “the plate data” whereas the preceding sentences of same paragraph and paragraph [0043] identify the data being acquired as “artwork data”. Replace “plate data” with “artwork data” to maintain consistent terminology.
Paragraph [0065] contains the browser-executable web address “https://arxiv.org/abs/2004.10934”. Applicant is required to delete this web address. The bibliographic citation to the YOLOv4 paper, including its authors, title, and arXiv identifier, may be retained. Any retained website reference should be limited to the domain name without a protocol prefix or browser-executable code. See MPEP §608.01, subsection VII.
Appropriate correction is required.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites “the first image data set including first image data” and “the second image data set including the first image data”. Claim 6 incorporates the limitations of claim 1 through claims 3-4. Claim 1 already recites “first image data” acquired from the captured image data and used to detect the first type region. It is unclear whether the first image data recited in claim 6 is the same first image data recited in claim 1, such that both training data sets must contain that inspection image data, or whether claim 6 introduces independently identified training image data common to the two training data sets.
The specification does not unambiguously resolve this distinction. Paragraph [0085] identifies first input image data used in inspection as an example of “first image data,” whereas paragraph [0138] uses that same expression for data of processed composite training image F1ax. Consequently, it is unclear whether claim 6 requires identity between the inspection image data and the common training image data. These interpretations impose different limitations on the claimed subject matter. Applicant should clarify the identity of the image data recited in claim 6.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 15 are rejected under 35 U.S.C. §102(a)(1) as being anticipated by Gutfinger (Gutfinger et al, US 2018/0365462 A1, 2018), as provided by Applicant's IDS filed on 10/31/2024.
Regarding claim 1, Gutfinger teaches
A non-transitory computer-readable storage medium storing a set of program instructions for a computer, the set of program instructions, when executed by the computer, causing the computer to perform:
( [0106], [0108–0111], [Fig. 18]: Storage 1803 includes non-transitory computer-readable media storing instructions executable by processors 1801 for image acquisition, label-region extraction, resolution adjustment, and label-adequacy assessment. )
acquiring captured image data of a captured image including a first object;
( [0073], [0110], [Fig. 12A]: The device acquires image 1205 depicting shipping label 1231, corresponding to the claimed first object. )
acquiring first image data from the captured image data, the first image data representing the first object with a first number of pixels;
( [0051], [0074], [Fig. 5 & 12A]: A reduced-resolution version of image 1205 is generated using a reduction factor of 8, representing label 1231 with a first pixel count. )
detecting a first type region indicating the first object by using the first image data;
( [0060], [0074], [Fig. 12A]: Shipping-label-candidate area 1234A is detected from the reduced-resolution image and encompasses label 1231. )
acquiring second image data from the captured image data, the second image data indicating a partial image, the partial image including the first type region of the captured image, the partial image not including at least part of a remaining region other than the first type region of the captured image;
( [0075], [Fig. 12A]: Region 1234A is expanded to region 1235, whose higher-resolution image is extracted from image 1205. This partial image retains region 1234A while excluding surrounding portions of image 1205. )
detecting a second type region by using the second image data, the second type region indicating at least part of the first object, the second image data representing the first object with a second number of pixels, the second number of pixels being greater than the first number of pixels; and
( [0075], [Figs. 12A–12B]: Region 1234B, indicating label 1231, is detected within the higher-resolution image of region 1235. Its factor-2 reduction represents the same label with more pixels than the first image’s factor-8 reduction. )
inspecting the first object by using a detection result of the second type region.
( [0076–0077], [Fig. 12B]: The device evaluates detected region 1234B for label shape, size, and barcode readability, determining label adequacy using the second detection result. )
Regarding claim 15, the rationale provided for claim 1 over Gutfinger is incorporated herein. Further, the non-transitory computer-readable storage medium of claim 1 corresponds to the data processing apparatus of claim 15 ( Gutfinger > [0105–0109], [0110–0113], [Fig. 18]: Gutfinger teaches barcode-reading device 1800 comprising one or more processors 1801, corresponding to the claimed controller, and storage 1803, corresponding to the claimed memory. Storage 1803 includes ROM and RAM and stores program instructions that, when executed by processors 1801, cause device 1800 to perform the image-acquisition, region-detection, resolution-adjustment, and label-inspection operations mapped above for claim 1. ). Therefore, the claim is rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3–6 are rejected under 35 U.S.C. §103 as being unpatentable over Gutfinger, in view of Tretter ( Tretter D, Bouman CA, Khawaja KW, Maciejewski AA. A multiscale stochastic image model for automated inspection. IEEE Transactions on Image Processing1995;4(12):1641-1654. doi:10.1109/83.475514.).
Regarding claim 3, Gutfinger teaches
The non-transitory computer-readable storage medium according to claim 1, wherein the first object includes N (N is an integer greater than or equal to 2) elements;
( [0073], [Figs. 12A–12B]: Shipping label 1231 corresponds to the first object and includes three barcodes 1232, which correspond to the N elements, where N = 3. )
wherein the detecting the second type region includes detecting, from the partial image, N second type regions indicating respective ones of the N elements; and
( [0039], [0068–0070], [0080], [Figs. 11, 14A–14B]: Gutfinger obtains a higher-resolution crop from the originally captured image. Its harvester identifies individual barcodes within an input label-candidate image and returns their respective values. Identifying the two barcode image areas implicitly teaches detecting respective constituent regions, each indicating part of the shipping label. This correspondence rests on the barcode-identification operation, rather than expressly disclosed region-coordinate outputs. )
wherein the inspecting includes inspecting the first object by comparing a positional relationship between the label-candidate area and the image boundary against a criterion for identifying a clipped label.
( [0060–0061], [0074], [Fig. 11, block B1110]: As part of determining whether a label-candidate area includes an adequate label, Gutfinger determines whether the candidate shares an image edge or lies very close to one. This evaluates the candidate’s positional relationship to the image boundary to determine whether the label is clipped. )
Gutfinger does not expressly disclose the constituent-region detection and inspection using expected spatial relationships, however Tretter teaches this:
wherein the inspecting includes inspecting the first object by comparing a positional relationship of the N second type regions with a particular positional relationship of the N elements.
( Tretter > [pp. 1642–1644, Figs. 2–3 and Eq. (2)], [p. 1646, Eq. (6)], [p. 1647, Fig. 7]: Tretter locates constituent image regions and models their relative positions through parent–child relationships. Equation (2) evaluates positional differences against expected offsets, adjusted for scale and rotation. Equation (6) incorporates that spatial comparison into the score used during constituent detection and inspection. Thus, the detected constituent arrangement is assessed against the modeled arrangement as part of inspecting the object. )
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to supplement Gutfinger’s barcode-identification and label-adequacy assessment with Tretter’s spatial assessment for labels having an expected printed layout. The reason would have been to use expected relative positions as additional evidence for identifying the barcode constituents and evaluating whether the label has the expected configuration. A reasonable expectation of success would follow from Tretter’s models of constituent appearance and relative position, trained on image examples and accommodating changes in scale and rotation. Training those models on examples of the selected label layout and applying them to the available higher-resolution crop would yield constituent-region detection and label inspection using the claimed positional comparison.
Regarding claim 4, Gutfinger [as modified by Tretter] teaches
The non-transitory computer-readable storage medium according to claim 3, wherein the detecting the N second type regions includes detecting, from the partial image, the N second type regions indicating respective ones of the N elements by using N element detection models, the N element detection models being trained to detect respective ones of the N elements.
( Tretter > [pp. 1642–1643, Figs. 1–3], [p. 1650, Fig. 9]; Gutfinger > [0068–0070], [0080]: Tretter separately models each constituent using learned appearance and spatial parameters. Applied to Gutfinger’s higher-resolution partial image, respective trained constituent models would detect the regions of the two barcode elements. The constituent-specific learned models correspond to the N element detection models. )
Regarding claim 5, Gutfinger [as modified by Tretter] teaches
The non-transitory computer-readable storage medium according to claim 4, wherein each of the N element detection models is trained by using image data indicating a plurality of elements, the plurality of elements including a corresponding element and an other element, the corresponding element being an element corresponding to one of the N element detection models for which training is performed, the other element being an element other than the corresponding element.
( Tretter > [p. 1647, Section IV, Eq. (10)], [p. 1651, Fig. 11]: Each constituent model is trained using assembly images containing its corresponding constituent and other constituents. In the Gutfinger [as modified by Tretter]'s system, each barcode model would likewise be trained using label images containing its corresponding barcode together with the other barcode. )
Regarding claim 6, Gutfinger [as modified by Tretter] teaches
The non-transitory computer-readable storage medium according to claim 4, wherein the N elements include a first element and a second element;
( Gutfinger > [0080], [Fig. 14A]: The shipping label contains two barcodes 1432. The one-dimensional barcode corresponds to the first element, and the two-dimensional barcode corresponds to the second element. )
wherein the N element detection models include a first element detection model for detecting the first element and a second element detection model for detecting the second element;
( Tretter > [pp. 1642–1643, Figs. 1–3], [p. 1650, Fig. 9]: Tretter provides separately trained constituent models. In the Gutfinger [as modified by Tretter]'s system, a first model would detect the first barcode element, and a second model would detect the second barcode element. )
wherein the first element detection model is trained by using a first image data set, the first image data set including first image data indicating a plurality of elements including the first element and the second element; and
( Tretter > [p. 1647, Eq. (10)], [p. 1651, Fig. 11]: The training set contains images depicting multiple constituents together. Applying this procedure, the first barcode model’s training set would include a label image depicting both barcode elements, corresponding to the claimed first image data. )
wherein the second element detection model is trained by using a second image data set, the second image data set including the first image data.
( Tretter > [p. 1647, Eq. (10)], [p. 1650, Fig. 9]: The same training images are used to estimate each constituent model’s parameters. Accordingly, the second barcode model’s training set would include the same label-image data used for the first model. The first and second sets may be identical; claim 6 does not require different sets. )
Claims 13–14 are rejected under 35 U.S.C. §103 as being unpatentable over Gutfinger [as modified by Tretter], in view of Richter (Richter et al, US 2015/0012171 A1, 2015).
Regarding claim 13, Gutfinger [as modified by Tretter] teaches
The non-transitory computer-readable storage medium according to claim 4, wherein the set of program instructions, when executed by the computer, causes the computer to perform:
determining whether all of the N second type regions have been detected by using the N element detection models;
( Tretter > [pp. 1642–1643], [p. 1647], [p. 1651, Fig. 12(a)]: Separately trained constituent models produce match/ no-match results, enabling determination whether all required constituents have been located. )
determining whether the combined scores for appearance and positional relationships between the N second type regions meet the acceptance criteria for all N constituents;
( Tretter > [pp. 1643–1644, Eq. (2)], [p. 1646, Eqs. (6), (9)], [p. 1647, Fig. 7]: The scores combine appearance matching with spatial likelihood based on learned expected relative positions and are tested against acceptance criteria. )
in response to determining that all of the N second type regions have been detected and determining that the combined scores for appearance and positional relationships between the N second type regions meet the acceptance criteria for all N constituents, determining that an inspection result is “passed”; and
( Tretter > [p. 1647, Fig. 7], [p. 1651, Fig. 12(a)]: The correctly assembled example passes when all constituents are located through accepted matches. )
in response to determining that at least one of the N second type regions has not been detected or determining that the positional relationships between the N second type regions are not correct, determining that an inspection result is “failed”.
( Tretter > [p. 1647], [p. 1651, Fig. 12(b)]: A search ending without a match declares the constituent missing; a nonmatching constituent causes the illustrated inspection to fail. )
Gutfinger [as modified by Tretter] does not expressly establish the claimed positional-correctness requirement and its specified effect on the inspection result. Richter supplies the additional completeness-and-correctness criteria, as follows:
determining whether positional relationships between the N second type regions are correct;
( Richter > [0022], [0054]: Richter determines whether all required components occupy their proper positions. In the combination, this correctness criterion applies to Tretter’s constituent-relative positions and their expected relationships. )
in response to determining that all of the N second type regions have been detected and determining that the positional relationships between the N second type regions are correct, determining that an inspection result is “passed”; and
( Richter > [0022–0023], [0054]: Complete and accurate inspected attributes receive passing results. Applying Richter’s completeness and placement requirements across the N constituent checks supplies the claimed passing condition. )
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to apply Richter’s completeness-and-correctness criteria to the constituent inspection performed by Gutfinger [as modified by Tretter], to reject labels having missing or incorrectly positioned required elements. The modification would pass the label when all required detections and positional conditions are satisfied and fail it when either requirement is unsatisfied. Success would have been reasonably expected because the existing combination supplies the constituent detections and relative-position information needed to implement those decision criteria.
Regarding claim 14, Gutfinger [as modifie by Tretter and Richter] teaches
The non-transitory computer-readable storage medium according to claim 13, wherein the positional relationships between the N second type regions include information indicating that one of the N second type regions is located above, below, left or right of another one of the N second type regions.
( Tretter > [pp. 1642–1644], [Figs. 2–3], [Eq. (2)]: Tretter locates constituent image regions by estimating their respective states, which include vertical and horizontal positions, and models each constituent’s position relative to its parent constituent. Differences between the estimated vertical and horizontal positions encode whether one constituent region lies above, below, left or right of another. Accordingly, applying Tretter’s relative-position representation to the detected element regions in the combination provides the claimed directional information. )
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEN KUDO whose telephone number is (571)272-4498. The examiner can normally be reached M-F 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KEN KUDO
Examiner
Art Unit 2671
/KEN KUDO/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights