DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The Applicant’s Remarks filed 02/04/2026 have been received and considered.
Claim 8 has been amended.
Claims 1 – 12, all of the claims pending in this application, have been rejected.
Response to Applicant’s Remarks
Applicant’s remarks were filed 02/04/2026, where there weren’t any amendments made to the independent claims. Referencing page 8 of Applicant’s Remarks, Applicant argues that Al Rashdan does not disclose “generating image data captured by the one or more image sensors by affine transformation”. However, Applicant has not provided any rationale or evidence to support this conclusory statement.
The Examiner disagrees with the remarks made by the Applicant. Examiner notes that the Applicant used a portion of the rejection made by the Examiner for the arguments, particularly Paragraph [0044]. Applicant did not seem to make any remarks in regards to the remaining portion of the rejection where the Examiner cited Paragraph [0033], which further disclosed how the images are transformed using a (2-D) transformation tool like MATLAB “affine2d”. It continues by explaining exactly why the affine transformation would be a good process to apply to the gauges as you can correct the way the image was captured. The abstract also discloses the rectification of images within the publication with the transformations being further described later in the application as the process used for the image correction at Paragraph [0094]. Figures 5 and 6 collaborates the previously disclosed paragraphs. These cited portions disclosed by Al Rashdan contradict the Applicant’s arguments and therefore the Examiner maintains that the combined prior art referenced in the non-final mailed 11/04/2025 do indeed teach the features of the claims, as detailed below.
Specification
Examiner notes that Paragraph [0063] and Paragraph [0066] of the Applicant’s Specification contradict one another with [0063] disclosing the pre-correction is performed by affine transformation, while [0066] contradicts this by disclosing that the affine transformation is performed on the pre-corrected data. It is known to those skilled in the art that performing an affine transformation is, in itself, the process of pre-correcting an image. Examiner suggests amending the specification as to delete the contradicting language of [0066] or adjust the language so that it reads congruently with [0063], but these changes are not mandatory.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 10 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 10 recites “performing affine transformation” on the “pre-corrected image data” which contradicts claim 8.
The specification discloses similar claim language to claim 8 in paragraph [0063] “The pre-correction unit 37 pre-corrects the target image data by affine transformation or homography transformation based on the two- dimensional shape of the two-dimensional barcode. The pre-correction unit 37 is an example of a pre-correction means.”, where it contradicts the claim language of claim 10 that states that affine transformation is performed on already pre-corrected image data. This specific process refers back to how it is claimed in the claim language in the specification in paragraph [0066], where it states “In the third present example embodiment, the generation unit 11 generates a transformed image group including a plurality of pieces of transformed image data by performing affine transformation on the target image data corrected by the pre-correction unit 37.” The specification in paragraphs [0067 – 0068], then states that the third embodiment, just mentioned in paragraph [0066], is described with reference to fig. 7; where fig. 7 detects the two-dimensional barcode. Paragraph [0069] of the specification then discloses “The pre-correction unit 37 pre-corrects the target image data by affine transformation or homography transformation based on the two- dimensional shape of the two-dimensional barcode (S302).”
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8 and 10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 8 and 10, as explained in the 112(a) rejection above, it is uncertain as to whether the pre-correction is performed before the affine transformation or if the affine transformation Is part of the pre-correction process. The specification does not clarify the matter as outlined above.
Examiner notes that performing affine transformation on an image is, in itself, a pre-correction process which is also known as image rectification.
Claim 10 is also rejected by virtue of its dependency on claim 8.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 7, 9, and 11 – 12 are rejected under 35 U.S.C. 103 as being unpatentable over (US Publication No. 2021/0142102 A1) to Al Rashdan et al. (hereinafter Al Rashdan) in view of Langlois et al. (US Publication No. 2018/0260940 A1) (hereinafter Langlois).
Claim 1
Regarding claim 1, an independent device claim, Al Rashdan teaches an image processing device comprising: a memory (Paragraph [0025]) configured to store instructions and at least one processor configured to perform the instructions to execute (Paragraph [0024]): generating a transformed image group including a plurality of pieces of transformed image data by performing affine transformation on target image data ("The one or more image sensors 108 are configured to generate image data 128 corresponding to one or more captured images 118 responsive to waves (e.g., optical waves) received at the one or more image sensors 108. By way of non-limiting example, the one or more image sensors 108 may be configured to capture image data 128 corresponding to one or more captured images 118 of the one or more panels 114. The one or more captured images 118 may be taken by the one or more image sensors 108 from any of a variety of positions and any of a variety of angles (e.g., oblique angles) to the one or more gauges 112a-112f.", Paragraph [0044]; "the image may be manually and iteratively transformed by a user to the point where a gauge in an image taken at an oblique angle appears to be circular for a circular gauge, or rectangular for a rectangular gauge. This process may also be performed using a two-dimensional (2-D) transformation tool like MATLAB “affine2d.” A 2-D affine transform would be a good process to apply to gauges installed on a flat panel because the entire oblique section of the image appears as a plane surface that may be effectively transformed in a single process. Processes like this may also be used to adjust satellite images taken at non-nadir (not straight down) conditions to geometrically correct images to match ground maps.", Paragraph [0035]), where this process is performed for each captured image;
calculating a correlation value between the transformed image data included in the transformed image group and master image data registered in advance ("For example, a Pearson correlation coefficient between a dial template at 360 evenly spaced rotations and the rectified and resized image being read may be calculated. The dial angle is obtained (e.g., in operation 214) as the angle with the greatest correlation coefficient. Then the angle of the gauge is determined (e.g., in operation 220) by correlated the meter background (e.g., a gauge template) with the rectified image. After the needle angle and gauge angle are known, then the value of the gauge can be estimated. A set of dial templates for 360 angles may be preprocessed by compositing high-quality dial templates on top of the meter background image template. The composited needle templates may be used to determine the dial angle by correlating the composited images with the rectified image at 360 evenly spaced rotations of the rectified use image.", Paragraph [0080]; "The template matches identified by the template matcher 1134 may be sorted by a sort template matches 1106, and the best candidate (e.g., determined by the highest correlation between a circle candidate determined by the circle detector 1126 and a gauge template 1132 including an image of the template gauge) is selected to be the rectified image 1130.", Paragraph [0130]);
correcting the selected transformed image data based on a positional deviation between an object appearing in the master image data and an object appearing in the selected transformed image data ("When template matching fails, the gauge reading method 200 may fall back to a perspective correction approach utilizing template matching (see FIG. 4). Template matching is a technique that utilizes empirically derived perspective relationships to identify the homography and rectify the use image. In some embodiments performing the template matching rectification on the use image includes: blurring at least one of a saturation and a value of the use image; thresholding the blurred use image according to one or more thresholds to provide one or more binary threshold images; detecting contours in the one or more binary threshold images; fitting ellipses to the detected contours in the one or more binary threshold images; and rectifying the use image based on the fit of the ellipses to the detected contours in the one or more binary threshold images to provide the rectified image. In some embodiments operation 218, performing the template matching rectification to produce the rectified image, includes a template matching rectification processing flow 400 discussed below with reference to FIG. 4.", Paragraph [0066]).
Although Al Rashdan teaches maximum values at Paragraph [0092] and [0131], selecting transformed image data having the correlation value being maximum from among the transformed image data included in the transformed image group is not explicitly taught;
However, Langlois teaches selecting transformed image data having the correlation value being maximum from among the transformed image data included in the transformed image group ("At operation 451, an affine transform is estimated for the image using image fiducials. For example, as illustrated in FIG. 7, bullseye ring fiducials 510 (light rings surrounded by a dark border to enhance contrast) may be found in the image to determine their actual locations in the image. In implementations, the locations of the fiducials in the image may be found by performing cross-correlation with the location of a reference virtual fiducial and taking the location where the cross-correlation score is maximized.", Paragraph [0078]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Al Rashdan to incorporate selecting transformed image data where the correlation score is maximized, as disclosed by Langlois. The suggestion/motivation for doing so would have been to use the image with the highest correlation score to perform more accurate analysis of the transformed image.
Claim 4
Regarding claim 4, dependent upon claim 2, Al Rashdan, in view of Langlois, teaches the invention as claimed in claim 2.
Al Rashdan further teaches wherein the pattern region is a region including a specific character or a specific number (Figure 11; "FIG. 11 is a QR code rectification processing flow 1100, according to some embodiments. In some instances a QR code may be positioned proximate to a gauge to enable electronic determination of properties of a gauge (e.g., an identification of the gauge, maximum and minimum gauge values, maximum and minimum gauge angles, etc.) responsive to the QR code. Since QR codes have a substantially square shape, image rectification may be performed based on rectifying the shape of the QR code to a square in addition to, or instead of, based on rectifying the shape of the gauge to a circle. QR code rectification may assume that the QR code and the gauge's face are parallel with each other in the image. The QR code rectification flow 1100 may also be applied to other information conveying features such as a barcode. The QR code rectification flow 1100 may further be applied to any known shape parallel to the face of the gauge. By way of non-limiting example, a white sticker of known shape (e.g., a square a rectangle or other shape) may be placed proximate to the gauge, and parallel to the face of the gauge.", Paragraph [0127]; "Conventional gauge reading technology does not have the ability to detect the location of multiple gauges contained within a single image. The gauge detection processing flow 1000 enables detection of gauges in an image and the resulting sub-images 1030 may be used in the gauge reading method 200 of FIG. 2 as use images. As a result, the gauge detection processing flow 1000 may take a captured image 1006 with multiple gauges of varying types, sizes, and colors, output a set of sub-images 1030 including the regions within the input image that may contain gauges that are readable by the gauge reading method 200 of FIG. 2.", Paragraph [0110]), where the gauges contain the QR code and identification information.
PNG
media_image1.png
520
715
media_image1.png
Greyscale
Claim 5
Regarding claim 5, dependent upon claim 1, Al Rashdan, in view of Langlois, teaches the invention as claimed in claim 1.
Al Rashdan does not teach wherein the at least one processor is configured to perform the instructions to execute: performing projective transformation of the selected transformed image data in such a way that positional coordinates of an object in the master image data are related to positional coordinates of the object in the selected transformed image data.
However, Langlois further teaches wherein the at least one processor is further configured to perform the instructions to execute: performing projective transformation of the selected transformed image data in such a way that positional coordinates of an object in the master image data are related to positional coordinates of the object in the selected transformed image data ("Given prior knowledge of the theoretical location of the fiducials (e.g., based on how many equally spaced spots there should be between the fiducials), an affine transform that maps the theoretical locations of the fiducials to their actual locations on the image may be determined. The estimated affine transform may map the translation, rotation, and magnification from the expected position of the fiducials.", Paragraph [0079]; "Given theoretical locations x.sub.i, y.sub.i of an image (i.e., where pixels of fiducials should be using the actual sample configuration) and actual image locations x.sub.w, y.sub.w (where pixels of fiducials actually appear on image), the affine transform may mathematically be represented", Paragraph [0078]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Al Rashdan, in view of Langlois, to incorporate positionally aligning the object in the transformed image with the same object in a master image, as disclosed by Langlois. The suggestion/motivation for doing so would have been to keep the image as much in correlation with each other to increase the accuracy of the expected outcome of further image analysis.
Claim 6
Regarding claim 6, dependent upon claim 1, Al Rashdan, in view of Langlois, teaches the invention as claimed in claim 1.
Al Rashdan further teaches wherein the at least one processor is further configured to perform the instructions to execute: reading a numerical value indicated by a pointer of an object from the corrected transformed image data (Figure 1; "FIG. 1 is a block diagram of a gauge reading system 100, according to some embodiments. The gauge reading system 100 includes one or more computing devices 116, a communication interface 110, one or more image sensors 108, and one or more panels 114 including one or more gauges 112a-112f. By way of non-limiting example, the one or more panels 114 may be one or more panels of a control room of a nuclear power plant. Many panels of nuclear power plants may include between substantially ten gauges and thirty gauges in a single panel. It will be understood, however, that the gauge reading system 100 may be used to read any gauges in any environment. Also by way of non-limiting example, the one or more gauges 112a-112f may include substantially circular gauges such as analog dial gauges, which may include glass fronts and pointer arms. As a further, non-limiting example, the one or more gauges 112a-112f may include gauges having shapes different from circular, such as rectangular.", Paragraph [0043]; "Circular scales may be detected by using circular arc detection methods extended to circle detection by using gradients. Such transformation algorithms may be used to detect circles and shapes. Also, image preprocessing may be used to improve image quality and reduce noise, enabling identification of the gauge pointer, detection of scale marks, and automatic recognition of the pointer gauge value. Essential elements of image preprocessing may include contrast, geometric adjustments, and filtering. A system may be used to detect multiple round gauges. This system may use a circular transformation (e.g., in MATLAB) to identify gauge edges and locate the gauge needle using the X-Y coordinates of the midpoint of the circle detected and the circle's radius.", Paragraph [0035]).
PNG
media_image2.png
513
717
media_image2.png
Greyscale
Claim 7
Regarding claim 7, dependent upon claim 6, Al Rashdan, in view of Langlois, teaches the invention as claimed in claim 6.
Al Rashdan further teaches wherein the at least one processor is configured to perform the instructions to execute: repeating generation of a predetermined number of pieces of transformed image data from the image data until a correlation value between at least one piece of transformed image data included in the transformed image group and master image data registered in advance exceeds a threshold value (Figure 2, "Conventional gauge reading technology does not have the ability to detect the location of multiple gauges contained within a single image. The gauge detection processing flow 1000 enables detection of gauges in an image and the resulting sub-images 1030 may be used in the gauge reading method 200 of FIG. 2 as use images. As a result, the gauge detection processing flow 1000 may take a captured image 1006 with multiple gauges of varying types, sizes, and colors, output a set of sub-images 1030 including the regions within the input image that may contain gauges that are readable by the gauge reading method 200 of FIG. 2", Paragraph [0110]), where the threshold dependent correlation values are discussed in claim 1 wherein a cross-correlation function is used.
PNG
media_image3.png
514
705
media_image3.png
Greyscale
Claim 9
Regarding claim 9, dependent upon claim 1, Al Rashdan, in view of Langlois, teaches the invention as claimed in claim 1.
Al Rashdan further teaches wherein the at least one processor is configured to perform the instructions to execute: repeating generation of a predetermined number of pieces of transformed image data from the image data until a numerical value indicated by a pointer of an object is successfully read from the corrected transformed image data (Figure 2; "For example, the image may be manually and iteratively transformed by a user to the point where a gauge in an image taken at an oblique angle appears to be circular for a circular gauge, or rectangular for a rectangular gauge. This process may also be performed using a two-dimensional (2-D) transformation tool like MATLAB “affine2d.” A 2-D affine transform would be a good process to apply to gauges installed on a flat panel because the entire oblique section of the image appears as a plane surface that may be effectively transformed in a single process.", Paragraph [0033]; "The ability to read and transform an analog value into a digital signal may be enabled using a combination of standard computer vision techniques to obtain an instrument reading from an image of its display that shows electrical quantities like voltage, current, power, and impedance…For analog instrumentation, the angle of the edge of the needle may be detected, and algebraic relationships are applied to associate the needle pointer with the value.", Paragraph [0034]).
Claim 2, dependent on claim 1, is rejected for the same reasons as applied to claim 1.
Claim 3, dependent on claim 1, is rejected for the same reasons as applied to claim 1, where the cross-correlation function is used to determine the peak pixel values and using the maximized scores for reference when determining correlating pixel locations between the images.
Claim 11, an independent method claim, is rejected for the same reasons as applied to claim 1.
Claim 12, an independent non-transitory medium claim, is rejected for the same reasons as applied to claim 1.
Claims 8 and 10 as best understood are rejected under 35 U.S.C. 103 as being unpatentable over (US Publication No. 2021/0142102 A1) to Al Rashdan et al. (hereinafter Al Rashdan) in view of Langlois et al. (US Publication No. 2018/0260940 A1) (hereinafter Langlois) in further view of Non-Patent Literature "Research on correction and recognition of QR code on cylinder" to Jin et al. (hereinafter Jin).
Claim 8
Regarding claim 8, dependent upon claim 1, Al Rashdan, in view of Langlois, teaches the invention as claimed in claim 1.
Al Rashdan further teaches wherein the at least one processor is further configured to perform the instructions to execute: detecting a two-dimensional barcode from the target image data (As best understood, "For example, the image may be manually and iteratively transformed by a user to the point where a gauge in an image taken at an oblique angle appears to be circular for a circular gauge, or rectangular for a rectangular gauge. This process may also be performed using a two-dimensional (2-D) transformation tool like MATLAB “affine2d.” A 2-D affine transform would be a good process to apply to gauges installed on a flat panel because the entire oblique section of the image appears as a plane surface that may be effectively transformed in a single process.", Paragraph [0033]; "Since QR codes have a substantially square shape, image rectification may be performed based on rectifying the shape of the QR code to a square in addition to, or instead of, based on rectifying the shape of the gauge to a circle. QR code rectification may assume that the QR code and the gauge's face are parallel with each other in the image. The QR code rectification flow 1100 may also be applied to other information conveying features such as a barcode. The QR code rectification flow 1100 may further be applied to any known shape parallel to the face of the gauge. By way of non-limiting example, a white sticker of known shape (e.g., a square a rectangle or other shape) may be placed proximate to the gauge, and parallel to the face of the gauge.", Paragraph [0127]).
Neither Al Rashdan, or Langlois, or the combination, teach pre-correcting the target image data by affine transformation or homography transformation
However, Jin (As best understood) teaches pre-correcting the target image data by affine transformation or homography transformation before performing the affine transformation, based on a two-dimensional shape of the two-dimensional barcode (Figures 3, 7, and 8; "In the actual application, the QR code was affected by the collection conditions, environment and surface of substrate, which would cause a series of defects such as noise pollution, local highlight and geometric distortion. These defects would lead to the reduction of recognition rate. The research of preprocessing, area detection, extraction and correction processing for these defects was based on the principle of image processing.", Abstract; "QR code were divided into two broad categories, the first category was stacked 2D barcodes and the other category was Matrix QR code.", Introduction; "The following major steps were performed. (1) First, the special position detection was acquired in QR code image, and recording the peripheral points when the first figure was as the module moving to the border in the horizontal direction. (2) The module unit was moving by the same token in the vertical direction. (3) The approaching curve were drawn with boundary points, and surface fitting figure was mapping according to the geometrical relation. The result of QR correction processing with QR code was shown in Fig. 3. (4) The corresponding coordinates in the distortion image was calibrated by gray interpolation, and smoothing the QR code image by closing operation of mathematical morphology.", "And the final correction processing image of QR code was gained by the affine transformation and antiPerspective transformation, and that of big curvature was used curve matching transformation to correct.").
PNG
media_image4.png
305
362
media_image4.png
Greyscale
PNG
media_image5.png
238
362
media_image5.png
Greyscale
PNG
media_image6.png
319
320
media_image6.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the teachings of Al Rashdan, in view of Langlois, to incorporate a correction process before applying the transformation, as disclosed by Jin. The suggestion/motivation for doing so would have been to make sure the image was aligned to ensure a more accurate extraction of the information.
Claim 10, dependent on claim 8, is rejected (As best understood) for the same reasons as applied to claim 1.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ronde Miller whose telephone number is (703) 756-5686 The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Gregory Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RONDE LEE MILLER/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698