DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 12 is objected to because of the following informalities: this claim appears to actually be dependent on claim 11, and for the purpose of examination, the examiner will treat the claim dependency in this way. Also, the limitation “while the remainder of the background member is not imaged in the thermal digital image because shielded by the foreground member” is recited. The examiner recommends amending the claim to recite “while the remainder of the background member is not imaged in the thermal digital image because the remainder of the background member is shielded by the foreground member”.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 4 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 4, the phrase "such as" renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 4, 5, 11, and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over D’huys et al. (Active Infrared Thermography for Seal Contamination Detection in Heat-Sealed Food Packaging, 2016, J. Imaging, Vol. 2, No. 33, Pages 1-19), hereinafter “D’huys”, in view of Ilhan et al. (Understanding the factors affecting the seal integrity in heat sealed flexible food packages: A review, 22 February 2021, Packag Technol Sci., Vol. 34, Pages 321-337), hereinafter “Ilhan”, in view of Li et al. (U.S. Pub. No. 2022/0214243), hereinafter “Li”.
Claim 1 is met by the combination of D’huys, Ilhan, and Li, wherein
D’huys discloses:
An automatic heat-sealing quality control system (See the Abstract.) to automatically control a quality of a heat-sealing of a sealing strip to a longitudinal edge of a packaging web at a heat-sealing station in a packaging machine, where sealed packages containing pourable food products are continuously produced from a continuous vertical tube filled with the pourable food product…(See Fig. 1 and the paragraph bridging pages 3-4: “Next, the film is formed into a tube, and the back seal or vertical seal is formed to close the tube of film. This tube of film is then transported downwards by means of drive belts, and the bottom seal of a bag is sealed with the horizontal seal bars.”)
the system comprises:
a sensory system arranged at the heat-sealing station to output an output that allows the quality of the heat-sealing of the sealing strip to the longitudinal edge of the packaging web to be evaluated (See the thermal camera in Fig. 1 and page 4, 1st paragraph under Fig. 1: “An IR camera (FLIR SC7600, FLIR Systems Inc., Wilsonville, OR, USA) was connected to the packaging machine.”); and
electronic computing resources designed to communicate with the sensory system to receive and process the output thereof to automatically evaluate the quality of the heat-sealing of the sealing strip to the longitudinal edge of the packaging web (See page 5, section 4.2, 2nd paragraph: “As a result of the way a flexible bag is formed in a VFFS machine, the thermal images of a recently-produced seal can be horizontally divided into three regions: one region in the center of the seal where an extra layer of polymer film is present due to the vertical seal and two regions located to the left and right of it.”);
the sensory system comprises an artificial vision system to capture digital images of the packaging web and of the sealing strip (See the thermal camera and its field-of-view of the packaging film on rollers and the three regions of seals in Fig. 1.);
the artificial vision system comprises:
…
a thermal imaging camera designed to operate in the electromagnetic spectrum invisible to human eye to capture and output thermal digital images (See page 4, 1st paragraph under Fig. 1: “An IR camera (FLIR SC7600, FLIR Systems Inc., Wilsonville, OR, USA) was connected to the packaging machine.” Also see page 2, last paragraph: “In this work, a packaging inspection system based on infrared (IR) thermography was studied. IR thermography is a contactless, non-destructive testing method in which surface temperature distributions (thermograms) are recorded based on the amount of IR radiation emitted by the inspected scene.”).
D’huys does not explicitly disclose the following; however, Ilhan discloses:
and formed by longitudinally folding the packaging web to overlap longitudinal edges thereof and then heat-sealing the overlapping longitudinal edges (See the folding of the packaging web in Fig. 3 on page 325, sections 3.3 & 3.4 for examples of folds, and especially section 3.6 on page 326: “In this type of sealing, the sealed edge will be parallel to the direction of the film movement. The side seals are produced by longitudinal sealing.” Finally see page 326, section 4.1: “Then the film is folded around the forming tube that was arranged to make a lap or fin seal at the longitudinal side of the package.”);
D'huys and Ilhan together partly disclose the limitations of claim 1. Ilhan is directed to a similar field of art (heat seal inspection). Therefore, D’huys and Ilhan are combinable. D’huys states that a vertical seal is formed to close the tube of film, but does not appear to describe longitudinally folding the web to overlap edges followed by heat-sealing the edges. Modifying the system and method of D’huys by adding the capability of “longitudinally folding the packaging web to overlap longitudinal edges thereof and then heat-sealing the overlapping longitudinal edges”, as disclosed by Ilhan, would yield the expected and predictable result of improved seal durability (fin seal) and reducing material waste (lap seal). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine D’huys and Ilhan in this way.
The combination of D’huys and Ilhan does not disclose the following; however, Li discloses:
the artificial vision system [that makes up the sensory system, which is arranged at the heat-sealing station] comprises:
a visible-light imaging camera designed to operate in the electromagnetic spectrum visible to human eye to capture and output visible-light digital images (See Fig. 6, photoelasticity vision system, and [0053]: “In an exemplary embodiment the first vision system is a thermography vision system comprising an infrared imaging device; and the second vision system is a photoelasticity vision system comprising:”. In [0216], the photoelasticity vision system is stated to output visible light digital images: “In an embodiment, the photoelasticity image capture device is an RGB camera capable of capturing images in the visible spectrum.”); and
a thermal imaging camera designed to operate in the electromagnetic spectrum invisible to human eye to capture and output thermal digital images (See Fig. 6, thermography vision system, and [0053]: “In an exemplary embodiment the first vision system is a thermography vision system comprising an infrared imaging device; and the second vision system is a photoelasticity vision system comprising:”.).
D'huys, Ilhan, and Li together disclose the limitations of claim 1. Li is directed to a similar field of art (in-line inspection of heat seals). Therefore, D’huys, Ilhan, and Li are combinable. In D’huys, the thermal camera is connected to the packaging/sealing machine, which meets the claimed “sensory system arranged at the heat-sealing station”. However, D’huys in Fig. 2 and page 4, last paragraph states that: “Next, the seal samples were retrieved from the VFFS machine, and high resolution digital images of the contaminated seals were acquired using an RGB-camera with a resolution of 4992 × 3328 pixels (Canon EOS-1 Ds Mark II Digital; Tokyo, Japan) equipped with a 28–105-mm zoom lens. A spatial resolution of 0.0417 mm per pixel was obtained. The images were recorded offline in a closed, cube-shaped box supplied with uniform illumination to avoid unwanted reflection of the ribbed profile present in the seals, as shown in Figure 2.” The RGB camera does not appear to meet the claimed “sensory system arranged at the heat-sealing station”. However, Li discloses a system and method in which both a thermal camera and RGB camera are in-line with a sealing mechanism (see [0143]: “FIG. 6 is a block diagram of an in-line seal defect detection system including a thermal vision system, a photoelasticity vision system and an ultraviolet vision system of an embodiment;”). Modifying the system and method of D’huys and Ilhan by simple substitution of the distributed camera setup for the in-line thermography & photoelasticity camera setup of Li would yield the expected and predictable result of a more compact imaging solution. Also, the problem identified by D’huys of unwanted reflections is mitigated by the use of polarizers of the photoelasticity vision system of Li. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine D’huys, Ilhan, and Li in this way.
Claim 2 is met by the combination of D’huys, Ilhan, and Li, wherein
The combination of D’huys, Ilhan, and Li discloses:
The system according to claim 1, wherein
And D’huys further discloses:
the visible-light and thermal imaging cameras are arranged with respect to the packaging web and the sealing strip to capture visible-light and thermal digital images of one and the same area of one and the same face of the packaging material and of the sealing strip (See the Abstract: “High resolution digital images served as a reference to quantify seal contamination, and processed thermal images were mapped to these references.” Also see the image registration in section 4.4.1 on page 8. Both passages imply that the thermal output images and digital images are of the same area of the film and vertical seal.).
Claim 4 is met by the combination of D’huys, Ilhan, and Li, wherein
The combination of D’huys, Ilhan, and Li discloses:
The system of claim 1, wherein the electronic computing resources are further configured to:
And D’huys further discloses:
process the received visible-light digital image in search of:
marks or patterns representative of macro defects or anomalies, such as scratches or delamination, that may form during the heat-sealing of the sealing strip to the packaging web (See the Abstract: “A common problem possibly jeopardizing seal quality is the presence of seal contamination, which can cause a decreased seal strength, an increased packaging failure risk and leak formation…In this work, a pulsed-type active thermography method using the heated seal bars as an excitation source was studied for detecting seal contamination.” Then see page 1, 2nd paragraph: “Seal contamination can cause a decreased seal strength and, thus, an increased risk of packaging failure [5].”), and
marks or patterns representative of micro defects or anomalies, such as wrinkles, that may form during the heat-sealing of the sealing strip to the packaging web (See page 1, 2nd paragraph: “Complete fusion of the seal layers can be hampered by an inadequate combination of those sealing parameters or by the presence of water vapor, air bubbles, wrinkles or product contamination in between the seal [4].”); and
evaluate the quality of the heat-sealing of the sealing strip to the packaging web also based on the identified marks or patterns (See page 8, section 4.4.3: “To compare the registered thermal output images to the digital reference images, two lists were created for each combination considered. The first list contains all contaminating particles identified in the binarized digital image and the pixel coordinates comprising those particles.”).
Claim 5 is met by the combination of D’huys, Ilhan, and Li, wherein
The combination of D’huys, Ilhan, and Li discloses:
The system of claim 4, wherein the electronic computing resources are further configured to:
And D’huys further discloses:
process the received visible-light digital image in search of marks or patterns representative of macro and micro defects or anomalies in a heat pattern of the packaging web defined as an area of the packaging web that is adjacent to the sealing strip and that was heated during heat-sealing of the sealing strip to the packaging web but not covered by the sealing strip (See page 5, section 4.2, 2nd paragraph: “As a result of the way a flexible bag is formed in a VFFS machine, the thermal images of a recently-produced seal can be horizontally divided into three regions: one region in the center of the seal where an extra layer of polymer film is present due to the vertical seal and two regions located to the left and right of it.”).
Claim 11 is met by the combination of D’huys, Ilhan, and Li, wherein
The combination of D’huys, Ilhan, and Li discloses:
The system of claim 1, wherein the electronic computing resources are further configured to
And D’huys further discloses:
align the geometrical information with the thermal information in the space domain, both in a direction parallel to the sealing strip and in a direction orthogonal thereto, such that the computed thermal information relates to the same spatial points of the packaging web and the sealing strip to which the computed visible-light information relates (See the image registration between the thermal image and digital image in section 4.4.1 on page 8.);
in order to align the geometrical information with the thermal information in the space domain, the electronic computing resources are further configured to compute a spatial transformation matrix which mutually relates the reference frames in which the geometrical and thermal information is referenced in the visible-light and thermal digital images, such that the sealing strip and the packaging web images in the visible-light digital image may be related to the sealing strip and the packaging web imaged in the thermal digital image (See );
in order for the spatial transformation matrix to be computed, the system comprises a calibration marker designed to exhibit a calibration pattern imageable both in a visible-light digital image and in a thermal digital image (See section 4.4.1: “Next, 10 pairs of control points were manually selected, each pair consisting of a location in the digital image and the corresponding location in the thermal output image. Based on these pairs of control points, the thermal output image was registered to the digital image using a projective transformation.”);
and the electronic computing resources are further configured to compute the spatial transformation matrix based on the position and orientation of the calibration pattern imaged both in a visible-light digital image and in a thermal digital image (See section 4.4.1: “Next, 10 pairs of control points were manually selected, each pair consisting of a location in the digital image and the corresponding location in the thermal output image. Based on these pairs of control points, the thermal output image was registered to the digital image using a projective transformation.”).
Claim 13 is met by the combination of D’huys, Ilhan, and Li, wherein
The combination of D’huys, Ilhan, and Li discloses:
The system of claim 10, wherein
And D’huys further discloses:
A packaging machine operable to continuously produce sealed packages containing pourable food products from a continuous vertical tube filled with the pourable food product and formed by longitudinally folding a packaging web to overlap longitudinal edges thereof and then heat-sealing the overlapping longitudinal edges; the packaging machine comprises an automatic heat-sealing quality control system as claimed in claim 1 (See the Abstract and Fig. 1.).
Allowable Subject Matter
Claims 3, 6-10, and 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: D’huys meets claim 3 as follows--
operate the visible-light and thermal imaging cameras to capture a pair of visible-light and thermal digital images of the packaging material and the sealing strip (See page 4, paragraph below Fig. 1: “trigger the IR camera to start recording thermal image sequences of the recently-produced seal.” Then see page 4, last paragraph: “high resolution digital images of the contaminated seals were acquired using an RGB-camera”.);
receive the pair of captured visible-light and thermal digital images (See page 5, 1st paragraph: “All image processing was performed in the 64-bit version of MATLAB 2013a (The MathWorks Inc., Natick, MA, USA) on a 3.40-GHz processor.” Section 4.4 that bridges pages 7-8 implies that the processor also receives corresponding digital reference images.);
carry out a visible-light digital image analysis of the received visible-light digital image to compute, and output data containing, geometrical information representative of the geometry of the sealing strip and of a heat pattern of the packaging web in a reference frame with an origin in an appropriate point of the processed visible-light digital image (See the binarization of the digital images in section 4.4.2 on page 8: “For the digital images, the threshold was calculated to the left of the peak, as the contamination had a lower intensity than the background.” Output of contaminating particles in the digital images meets the claimed “geometric information”. See the processed images in Fig. 4 which meet the claimed “reference frame with an origin in an appropriate point”.);
carry out a thermal digital image analysis of the received thermal digital image to compute, and output data containing, thermal information representative of a temperature profile of the packaging web and of the sealing strip in a direction orthogonal to the packaging web and the sealing strip in a reference frame with an origin in an appropriate position of the processed thermal digital image (See the binarization of the thermal images in section 4.4.2 on page 8: “For all thermal output images, the threshold was calculated to the right of the peak, as the contamination had a higher intensity than the background.” Output of contaminating particles in the thermal images meets the claimed “thermal information representative of a temperature profile”. See the processed images in Fig. 4 which meet the claimed “reference frame with an origin in an appropriate point”.);
However, D’huys does not clearly (see section 4.4.3 on page 8) disclose the following:
fuse the geometrical information obtained from the visible-light digital image analysis with the thermal information obtained from the thermal digital image analysis to compute fused data representative of the fused information;
process the fused data to identify underheating or cold seal or overheating, if any, that may have occurred during the heat-sealing of the sealing strip to the packaging web; and
evaluate the quality of the heat-sealing of the sealing strip to the packaging web based on the identified underheating or overheating.
Li discloses separately processing thermal and photoelasticity data. See [0288]: “These multiple different forms of image data for the same seal may be passed through an artificial intelligent algorithm separately. If the artificial intelligent algorithm returns the same classified state for the seal using multiple different forms of image data, then the confidence level of the classification for that seal can be increased significantly. In one example, if the artificial intelligent algorithm classified one of the images as having a packaging article with an imperfect seal at a 98% confidence level and classified the other image as having a packaging article with an imperfect seal at a 96% confidence level, then the confidence level that the packaging article has an imperfect seal may be greater than 99%.”
In addition, the cited prior art of record does not disclose or suggest in:
-Claim 6
process the received visible-light digital image in search of marks or patterns representative of micro defects or anomalies comprising:
diagonal wrinkles in the heat pattern of the packaging web; and
transversal wrinkles in either one or both of the sealing strip and the heat pattern of the packaging web.
-Claim 7
compute an actual heat pattern width, measured in a direction orthogonal to the sealing strip of a heat pattern representing an area of the packaging web that is adjacent to the sealing strip and that was heated during heat-sealing of the sealing strip to the packaging web but not covered by the sealing strip; and
evaluate the quality of the heat-sealing of the sealing strip to the packaging web also based on the computed heat pattern width.
-Claim 8
compute an actual sealing strip overlap, measured in a direction orthogonal to the sealing strip, indicative of an extent the sealing strip actually overlaps the packaging web; and
evaluate the quality of the heat-sealing of the sealing strip to the packaging web also based on the computed actual sealing strip overlap.
-Claim 9
determine whether the sealing strip is U-folded around a longitudinal edge of the packaging web and marks similar to fishbones have formed in the sealing strip during U-folding and heat-sealing of the sealing strip to the packaging web;
if the sealing strip is determined to be U-folded, compute an actual sealing strip overlap, measured in a direction orthogonal to the sealing strip, indicative of an extent the sealing strip actually overlaps the packaging web; and
evaluate the quality of the heat-sealing of the sealing strip to the packaging web also based on the determination as to whether the sealing strip is U-folded around a longitudinal edge of the packaging web, on whether marks similar to fishbones have formed in the sealing strip during U-folding and heat-sealing of the sealing strip to the packaging web, and on the computed actual sealing strip overlap.
-Claim 10 is dependent on claim 9 and includes the same allowable subject matter.
-Claim 12
the calibration marker comprises a foreground member intended to be arranged in the foreground with respect to the visible-light and thermal imaging cameras and carrying the calibration pattern in the form of a through-passing pattern; and a background member intended to be arranged behind the foreground member with respect to the visible-light and thermal imaging cameras so as to be exposed in the foreground through the calibration pattern;
the foreground member and the background member are made of materials with different emissivity whereby, the calibration pattern formed in the foreground member may be imaged in a visible-light digital image and, when the background member is heated, the heated areas of the background member exposed in the foreground through the calibration pattern formed in the foreground member may be imaged in a thermal digital image and forms a corresponding calibration pattern, while the remainder of the background member is not imaged in the thermal digital image because shielded by the foreground member.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN S LEE whose telephone number is (571)272-1981. The examiner can normally be reached 11:30 AM - 7:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jonathan S Lee/Primary Examiner, Art Unit 2677
Prosecution Insights