DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 is objected to because of the following informalities: Claim 1, line 8, the term “comparting” appears to be misspelled. Examiner believes the term “comparting” should be corrected to be –comparing--. Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-8 and 10-11 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chauhan et al., “Fault detection and classification in automated assembly machines using machine vision”.
Regarding claim 1, Chauhan discloses an operation monitoring method of monitoring an operation of a processing unit (Abstract; Section 5.2; methods of fault detection optical flow in automated assembly machines),
the operation monitoring method comprising:
a) acquiring a plurality of moving images by shooting, a plurality of times, a specific operation of the processing unit (Abstract; Introduction, Third paragraph; Section 5; These systems use industrial grade cameras to acquire data in the form of images and videos. A camera can be used for continuous video acquisition of the machine’s operation);
b) calculating a motion vector for each pixel by an optical flow for a plurality of frame images included in the plurality of moving images (Fig. 13; Section 5.2; Optical
flow was introduced …for computing motion between two video frames. Optical flow for a pixel between two video frames is shown in Fig. 13. It is a 2D vector that represents motion in horizontal and vertical directions. For each pixel, it is plotted as a line joining the pixels current position with its previous position. The optical flow density (OFD) (i.e., motion vector) is computed from the optical flow of the video. The OFD, the sum of the absolute values of optical flow vectors, is a single number that represents the motion in a frame);
c) calculating an evaluation value based on the motion vector for each pixel by comparting the motion vectors calculated for the same pixel at the same time in the plurality of moving images (Fig. 13; Section 5.2; Optical flow for a pixel between two video frames is shown in Fig. 13. It is a 2D vector that represents motion in horizontal and vertical directions. For each pixel, it is plotted as a line joining the pixels current position with its previous position. During a normal operation sequence, the OFD stays within limits (between zero and the threshold). The OFD of the normal operation sequence is considered as the threshold (reference value). For each frame, the OFD is determined and compared with the threshold. If the current frame OFD exceeds the threshold and keeps on increasing with respect to time, then that is the indication of more number of objects in the frame. The method then keeps track of the OFD for next few frames, and if the trend continues, the machine condition is considered as a fault); and
d) evaluating the specific operation of the processing unit in accordance with the evaluation value (Fig. 14; Section 5.2; During a normal operation sequence, the OFD stays within limits (between zero and the threshold). The OFD of the normal operation sequence is considered as the threshold (reference value). For each frame, the OFD is determined and compared with the threshold. If the current frame OFD exceeds the threshold and keeps on increasing with respect to time, then that is the indication of more number of objects in the frame. The method then keeps track of the OFD for next few frames, and if the trend continues, the machine condition is considered as a fault. The OFD is the absolute value of optical flow as determined for each frame. The OFD is used as a feature to detect and classify faults from videos).
Regarding claim 2, the operation monitoring method according to claim 1, Chauhan discloses wherein the evaluation value is a standard deviation of the motion vector obtained by using, as a population, the motion vectors calculated for the same pixel at the same time in the plurality of moving images (Section 5.1.3).
Regarding claim 3, the operation monitoring method according to claim 2, Chauhan discloses wherein
the plurality of frame images are two-dimensional images defined by an x axis and a y axis (Fig. 13; Section 5.2),
the operation c) includes calculating a standard deviation of an x-axis component of the motion vector and a standard deviation of a y-axis component of the motion vector (Fig. 13; Sections 5.1.3 and 5.2), and
in the operation d), the specific operation of the processing unit is evaluated in accordance with an average value of the standard deviation of the x-axis component and the standard deviation of the y-axis component (Fig. 13; Sections 5.1.3 and 5.2).
Regarding claim 4, the operation monitoring method according to claim 1, Chauhan discloses further comprising:
x) calculating a feature based on the motion vector for each of the plurality of frame images in each of the plurality of moving images (Figs. 13-14; Section 5.2); and
y) synchronizing timing of the plurality of moving images with one another in accordance with a time-varying waveform of the feature (Figs. 13-14; Section 5.2),
the operation x) and the operation y) being performed after the operation b) and before the operation c) (Fig. 14; Section 5.2).
Regarding claim 5, the operation monitoring method according to claim 4, Chauhan discloses wherein
the feature is an average value of lengths of the motion vectors for a plurality of pixels included in the plurality of frame images (Sections 5.2 and 6.2).
Regarding claim 6, the operation monitoring method according to claim 1, Chauhan discloses wherein
the operation d) includes, when the evaluation value deviates from a preset tolerance, outputting an evaluation result indicating that the specific operation exhibits a wide range of variations (Section 5.2).
Regarding claim 7, the operation monitoring method according to claim 1, Chauhan discloses wherein
in the operation a), the plurality of moving images are acquired by shooting, the plurality of times, the specific operation for each of a processing unit that serves as a reference and a different processing unit (Section 5, Second paragraph), and
the operation d) includes, when a difference between the evaluation value calculated for the processing unit serving as the reference and the evaluation value calculated for the different processing unit deviates from a preset tolerance, outputting an evaluation result indicating that the specific operation of the different processing unit exhibits a wide range of variations (Section 5, Second paragraph and Section 5.2).
Regarding claim 8, the operation monitoring method according to claim 6, Chauhan discloses wherein
in the outputting of the evaluation result, a pixel whose evaluation value deviates from the preset tolerance in the plurality of frame images is displayed with a color, a character, or a graphic overlaid on the pixel (Figs. 14-15; Section 5.2; output feature plots and frames).
Regarding claim 10, this claim recites substantially the same limitations that are performed by claim 1 above, and it is rejected for the same reasons.
Regarding claim 11, this claim recites substantially the same limitations that are performed by claim 8 above, and it is rejected for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chauhan et al., “Fault detection and classification in automated assembly machines using machine vision” in view of Choi et al., US 2020/0122165.
Regarding claim 9, the operation monitoring method according to claim 1, Chauhan discloses wherein the processing unit is a unit that supplies a substrate (Abstract; Section 3; O-ring).
Chauhan discloses claim 9 as enumerated above, but Chauhan does not explicitly disclose supplies a processing liquid to a surface of a substrate as claimed.
However, Choi discloses the first nozzle may dispense the first processing liquid onto a top side of the substrate (para 0023).
Therefore, taking the combined disclosures of Chauhan and Choi as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the first nozzle may dispense the first processing liquid onto a top side of the substrate as taught by Choi into the invention of Chauhan for the benefit of treating a substrate (Choi: Abstract).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Toyoda, US 2019/0104313 discloses a motion vector calculation method for calculating a motion vector of a pixel between image frames included in a moving image or a still image group photographed continuously.
Wang et al., US 2017/0261264 discloses a fault diagnosis method for an electrical fused magnesia furnace.
Jones et al., US 2017/0255832 discloses a method and system detects actions of an object in a scene by first acquiring a video of the scene as a sequence of images, wherein each image includes pixels, wherein the video is partitioned into chunks.
Liu et al., US 2019/0209113 discloses a method, storage medium, and system for analyzing an image sequence of a periodic physiological activity.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN D HUYNH whose telephone number is (571)270-1937. The examiner can normally be reached 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN D HUYNH/Primary Examiner, Art Unit 2665
Prosecution Insights