DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/30/2026 has been entered.
Response to Arguments
Applicant's arguments, see Remarks pages 7-12, filed 01/30/2026, with respect to the claim rejections of amended claims 1, 3, and 16-17 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
On page 8-9 of Remarks, Applicant argues:
PNG
media_image1.png
1146
747
media_image1.png
Greyscale
Examiner respectfully disagrees.
Page 3133 Col 1 of Dasgupta discloses “The output from the FRB is passed to the RoIRotate [2] after passing through DRecog consisting of a series of deconvolution layers. DRecog reduces the number of input feature maps while increasing their resolution to facilitate robust recognition of small sized texts. RoIRotate also receives the RBOX geometry information from the text detection branch. It horizontally aligns the upsampled feature map obtained from DRecog by rotating it using the orientation angle of the RBOX information and also resizes it. This output of RoIRotate is passed to the Text recognition branch to predict the sequence of characters in the detected text box.” Thus, a detected textbox is extracted from the image and performs a correction of the extracted textbox prior to the recognition of the text within the textbox. Wherein as disclosed in Figure 1 and page 3130 Col 1 of Dasgupta, the text spotting results of Dasgupta’s framework produce uniform results of the detected text.
Thus, Haase, hereinafter referenced as Ohrt, in view of Dasgupta discloses the limitation “the at least one computing device transforming image data of the at least one text region relative to remaining image data outside of the at least one text region to calculate a transformed text region using the determined at least one geometric property so that the transformed text region depicts the text with a desired geometric property.”
However, the limitation “creating an edited overview image in which the at least one text region is replaced by the transformed text region which has been transformed relative to the remaining image data of the overview image,” is disclosed by Ohrt in view of Dasgupta and Wei, as is further disclosed in arguments and the rejection of claim 1 under 35 U.S.C. 103 below.
On page 11 of Remarks, Applicant argues:
PNG
media_image2.png
394
746
media_image2.png
Greyscale
Examiner respectfully disagrees.
As disclosed above and in the rejection of claim 1 under 35 U.S.C. 103 below, as Ohrt, in view of Dasgupta discloses the limitation “the at least one computing device transforming image data of the at least one text region relative to remaining image data outside of the at least one text region to calculate a transformed text region using the determined at least one geometric property so that the transformed text region depicts the text with a desired geometric property.”
In addition, paragraph 0026 of Wei discloses “FIG. 2 shows an exemplary flow of operation for the orientation, estimation, and reconstruction of the curved or otherwise non-parametric surface ( 130 in FIG. 1). At 210 , the surface is scanned, to acquire geometrical data with RGB (color) data as well as the text to be modified (old text or target text). At 220 , the scanned surface is discretized by being broken up into distinct parts. At 230 , those distinct parts are treated as segments, to facilitate creation of a mapping of segments between old or target text, on the one hand, and candidate text, on the other. At 240 , the background of the surface is identified, and is filled out so that when the new text is placed on that background, there are no inconsistencies or missing pieces of background. One technique for filling out the background is image inpainting, which recovers the target text background for the placement of the candidate text. At 250 , once the surface has been discretized and segmented, text may be resampled to get a more accurate rendering of the text, thereby facilitating accurate replacement of that text with new text,” wherein a new text can replace an old text present in the image through the use of surface segmentation and image inpainting.
Thus, it would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to implement the algorithms for text replacement taught by Wei into Ohrt in view of Dasgupta by replacing the text regions with their corresponding transformed counterpart.
Therefore, Ohrt in view of Dasgupta and Wei discloses “the at least one computing device transforming image data of the at least one text region relative to remaining image data outside of the at least one text region to calculate a transformed text region using the determined at least one geometric property so that the transformed text region depicts the text with a desired geometric property, creating an edited overview image in which the at least one text region is replaced by the transformed text region which has been transformed relative to the remaining image data of the overview image.”
As per claim(s) 16 & 17, arguments made in rejecting claim(s) 1 are analogous.
On page 10 of Remarks, Applicant argues:
PNG
media_image3.png
783
770
media_image3.png
Greyscale
Examiner respectfully disagrees.
Page 3134 Col 2 of Dasgupta discloses “Data Augmentation is also employed to improve the robustness of the proposed framework. For data augmentation, we make use of random mirror and resize between 0.5 and 2 for all datasets, additional we add some random rotation between -10 to 10 degrees, Gaussian blur, brightness and contrast for each sample of the dataset. This comprehensive data augmentation scheme makes the network to resist overfitting and improve accuracy.” Wherein the augmented training dataset images include mirrored images.
Page 3132 Col 2 of Dasgupta discloses “This part of the network, designed for instant scene text detection, consists of three Conv1_1 layers operated in parallel generating the output consisting of a single-channel score map and the RBOX geometry which in turn consists of 4 channels for each text box and a single channel towards its orientation angle,” wherein the framework performs text detection on the input image in order to extract textboxes within the image, which are further processed by the Text Recognition branch in order to predict the text present in the textboxes, as is disclosed in page 3133 Col 1 of Dasgupta. Thus, the text output by the trained model’s Text Recognition branch disclosed by Dasgupta constitutes mirror corrected text corresponding to the mirror-reversed text regions in the image.
Therefore, Ohrt in view of Dasgupta and Wei discloses the limitations “wherein, in cases where the text is mirror-reversed, the creating the edited overview image includes mirroring of the at least one text region without mirroring the remaining image data outside of the at least one text region so that a part of the overview image is mirrored relative to the remaining image data of the overview image and a mirror-reversed text orientation is not present in the transformed text region.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-5, 7-11, and 13-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ohrt et al. (US2020088984A1) hereinafter referenced as Ohrt, in view of Dasgupta et al. (Stratified Multi-Task Learning for Robust Spotting of Scene Texts) hereinafter referenced as Dasgupta, and Wei et al. (US2021097323A1) hereinafter referenced as Wei.
Regarding claim 1, Ohrt discloses: A computer-implemented method for editing overview images of a microscope (Ohrt: Abstract), comprising: receiving, on at least one computing device, a raw overview image of a sample carrier generated by an overview camera of a microscope (Ohrt: Figure 1; 0059: “The detection optical unit 2 can be a microscope objective having a small magnification, although it is preferably the objective lens of a camera, which is able to image a larger region of an object field, which then ideally covers the entire sample carrier 1 in an overview.”);
the at least one computing device transforming the raw overview image with calibration data in order to calculate an overview image in which image directions coincide with given directions (Ohrt: 0014-0015: “A calculation algorithm that is used to calculate an overview contrast image from the at least two overview raw images is chosen in dependence on the type of the illumination and information that is to be extracted from the overview contrast image…an image evaluation calculation algorithm is chosen that is used to extract the information from the overview contrast image. Said information can then be used for example by the user on the screen of a connected computer to initiate further steps as part of the observation and analysis, for example to navigate on the sample, which is accomplished by the image being represented on a screen and the user choosing the sample region of interest for example by way of a mouse click.”; Wherein the raw overview images are converted based on calibration data into contrast images used for navigating based on a user’s given directions);
the at least one computing device determining at least one geometric property of text of at least one text region in the overview image (Ohrt: 0067: “a sample number is indicated on the sample carrier 1, for example by way of a handwritten inscription, but more frequently as a code with a barcode or QR code,”; 0090: “The calibration pattern described above in connection with the calibration of the relative movement can additionally be used to effect a correction of geometric distortions in the image, applied to each overview contrast image. In addition, it is also possible to eliminate background artefacts by calculation.”; Wherein the calibration pattern is used to correct determined distortions present on overview images).
Ohrt does not disclose expressly: the at least one computing device transforming image data of the at least one text region relative to remaining image data outside of the at least one text region to calculate a transformed text region using the determined at least one geometric property.
Dasgupta discloses: the at least one computing device transforming image data of the at least one text region relative to remaining image data outside of the at least one text region to calculate a transformed text region using the determined at least one geometric property (Dasgupta: Figures 2-3; Page 3133: Col 1: “The output from the FRB is passed to the RoIRotate [2] after passing through DRecog consisting of a series of deconvolution layers. DRecog reduces the number of input feature maps while increasing their resolution to facilitate robust recognition of small sized texts. RoIRotate also receives the RBOX geometry information from the text detection branch. It horizontally aligns the upsampled feature map obtained from DRecog by rotating it using the orientation angle of the RBOX information and also resizes it. This output of RoIRotate is passed to the Text recognition branch to predict the sequence of characters in the detected text box...”; Wherein the image text boxes are extracted, predicted, and displayed next to the detected text.).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the learning framework for the recognition of scene text disclosed by Dasgupta for the recognition and transformation of text regions disclosed by Ohrt. The suggestion/motivation for doing so would have been “Various sub-tasks performed by this network includes text detection, text segmentation and text recognition…Usually, such a multi-task framework developed using a deep architecture has a common feature encoding part…task specific features cannot be preserved into its feature encoder which in turn may affect the performance of the framework. The Feature Representation Block placed at the top of the feature encoding component of the network should take care of this issue.” (Dasgupta: Page 3136: Col 2). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Ohrt in view of Dasgupta does not disclose expressly: creating an edited overview image in which the at least one text region is replaced by the transformed text region which has been transformed relative to the remaining image data of the overview image.
Wei discloses: creating an edited overview image in which at least one text region is replaced by a transformed text region which has been transformed relative to a remaining image data of the overview image (Wei: Figure 8; 0026: “At 240, the background of the surface is identified, and is filled out so that when the new text is placed on that background, there are no inconsistencies or missing pieces of background.”; 0035: “In one aspect, the background can be filled in using an image inpainting technique so that the new text appears over the background, without differently-colored spaces suggesting deletion and insertion of text. Mapping shown at 860 mirrors the mapping done at 820 so that the replacement text appears naturally in place of the original text.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the algorithms for text replacement taught by Wei into Ohrt in view of Dasgupta by replacing the text regions with their corresponding transformed counterpart. The suggestion/motivation for doing so would have been “the background can be filled in using an image inpainting technique so that the new text appears over the background, without differently-colored spaces suggesting deletion and insertion of text” (Wei: 0035). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ohrt in view of Dasgupta with Wei to obtain the invention as specified in claim 1.
Regarding claim 3, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein it is determined as a geometric property of the text whether the text is mirror-reversed, wherein, in cases where the text is mirror-reversed, the creating the edited overview image includes mirroring of the at least one text region without mirroring the remaining image data outside of the at least one text region so that a part of the overview image is mirrored relative to the remaining image data of the overview image (Dasgupta: Figure 2; Page 3131: Col 1: “Contributions of the present study are as follows. 1) A novel end-to-end trainable framework towards robust detection, segmentation and recognition of scene texts.”; Page 3134: Col 2: “For data augmentation, we make use of random mirror and resize between 0.5 and 2 for all datasets,”; Wherein for data augmentation the text is mirrored and the proposed model reverses the mirroring) and a mirror-reversed text orientation is not present in the transformed text region (Wei: Figure 8; 0026: “At 240, the background of the surface is identified, and is filled out so that when the new text is placed on that background, there are no inconsistencies or missing pieces of background.”; 0035: “In one aspect, the background can be filled in using an image inpainting technique so that the new text appears over the background, without differently-colored spaces suggesting deletion and insertion of text. Mapping shown at 860 mirrors the mapping done at 820 so that the replacement text appears naturally in place of the original text.”; Wherein the originally mirrored text is replaced by the corrected text).
Regarding claim 4, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein a rotational orientation is determined as a geometric property of the text, wherein the calculating of the transformed text region comprises a rotation of the text region relative to adjacent image content so as to provide a certain rotational orientation in the transformed text region (Dasgupta: Figures 1 and 2; Page 3131: Col 1: “Contributions of the present study are as follows. 1) A novel end-to-end trainable framework towards robust detection, segmentation and recognition of scene texts.”; Page 3134: Col 2: “For data augmentation…additional we add some random rotation between -10 to 10 degrees, G,”; Wherein for data augmentation the text is rotated and the proposed model reorientates the text.).
Regarding claim 5, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein a perspective distortion is determined as a geometric property of the text, wherein the calculating of the transformed text region comprises a perspective rectification of the text region (Dasgupta: Figure 3; Page 3134: Col 1-2: “ICDAR 2015 [14] dataset includes 1000 training images and 500 testing incidental scene images used for oriented scene text detection and spotting. Text in the scene can be in arbitrary orientations or suffer from motion blur and low resolution. ICDAR 2017 MLT [41] is a large multi-lingual, multioriented, multi-script text dataset, includes 7200 training images, 1800 validation images and 9000 testing images. MSRA-TD500 [42] is a dataset comprises 300 training images and 200 test images. Text areas are arbitrary orientations including line level annotations, contains text in both English plus Chinese”; Wherein the multi-oriented text present in the datasets, includes distorted text, as shown by Figure 3.).
Regarding claim 7, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein calibration data is used in at least one of the determining of the geometric property or the calculating of the transformed text region (Ohrt: 0090: “The calibration pattern described above in connection with the calibration of the relative movement can additionally be used to effect a correction of geometric distortions in the image, applied to each overview contrast image.”; Wherein the calibration pattern is evaluated in order to correct geometric distortions in the overall overview image.).
Regarding claim 8, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein characters of the text in the text region are identified using optical character recognition (OCR) (Dasgupta: Page 3131: Col 2: “The main objective of the text recognition system is the conversion of the variable-length cropped text images into machine-encoded text.”) and wherein the transformed text region is generated by replacing the text with newly generated characters corresponding to the characters identified with OCR but differing in their arrangement so as to satisfy the desired geometric property (Wei: Figure 8; 0035: “FIG. 8 shows how the new text (in this example, Japanese text) replaces original text (I this example, English text) on a curved surface. At 810, there is a mapping of the coordinates of the endpoints of the background to be manipulated. At 820, the words “Konica Minolta” are located on a background. 830 shows the replacement text as a combination of text 840 and background 850. In an embodiment, once the replacement text is determined, that text can be mapped to the background according to the mapping done at 820.”; Wherein the distorted text is detected by the text recognition module disclosed by Dasgupta, corrected, and is used to replace the text present in image).
Regarding claim 9, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 8, wherein the newly generated characters generated by OCR are saved in the overview image as an additional text layer in order to enable further machine- readable text processing (Wei: 0035: “In one aspect, the background can be filled in using an image inpainting technique so that the new text appears over the background, without differently-colored spaces suggesting deletion and insertion of text.”; Wherein the distorted text is detected by the text recognition module disclosed by Dasgupta, corrected, and placed a layer over the background)
Regarding claim 10, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein the edited overview image is saved or displayed on a screen in addition to the overview image containing the text region, or wherein the text region in the overview image is replaced by the transformed text region (Dasgupta: Figures 2-3; Page 3133: Col 1: “The output from the FRB is passed to the RoIRotate [2] after passing through DRecog consisting of a series of deconvolution layers. DRecog reduces the number of input feature maps while increasing their resolution to facilitate robust recognition of small sized texts. RoIRotate also receives the RBOX geometry information from the text detection branch. It horizontally aligns the upsampled feature map obtained from DRecog by rotating it using the orientation angle of the RBOX information and also resizes it. This output of RoIRotate is passed to the Text recognition branch to predict the sequence of characters in the detected text box...”)
(Wei: Figure 8; 0035: “FIG. 8 shows how the new text (in this example, Japanese text) replaces original text (I this example, English text) on a curved surface. At 810, there is a mapping of the coordinates of the endpoints of the background to be manipulated. At 820, the words “Konica Minolta” are located on a background. 830 shows the replacement text as a combination of text 840 and background 850. In an embodiment, once the replacement text is determined, that text can be mapped to the background according to the mapping done at 820.”; Wherein the distorted text is detected by the text recognition module disclosed by Dasgupta, corrected, and is used to replace the text present in image).
Regarding claim 11, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein, in cases where the transformed text region has a different size or shape than the text region so that a gap occurs when the text region is replaced by the transformed text region, the gap is filled with a model trained for image reconstruction (Wei: 0026: “At 240, the background of the surface is identified, and is filled out so that when the new text is placed on that background, there are no inconsistencies or missing pieces of background. One technique for filling out the background is image inpainting, which recovers the target text background for the placement of the candidate text.”; Wherein the algorithm used for image inpainting constitutes a trained model).
Regarding claim 13, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein the text in the overview image is removed using a model trained for image reconstruction and the transformed text region is inserted thereafter (Wei: Figure 8; 0026: “At 240, the background of the surface is identified, and is filled out so that when the new text is placed on that background, there are no inconsistencies or missing pieces of background.”; 0035: “In one aspect, the background can be filled in using an image inpainting technique so that the new text appears over the background, without differently-colored spaces suggesting deletion and insertion of text. Mapping shown at 860 mirrors the mapping done at 820 so that the replacement text appears naturally in place of the original text.”).
Regarding claim 14, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein the transforming of the raw overview image is based on a homography estimation or involves a perspective distortion of the raw overview image (Ohrt: 0086: “…initially a calibration pattern is used instead of the sample carrier at the same position or clamped onto the stage. In this way, it is possible to estimate such a mapping—a homography, that is to say a mapping of a two-dimensional plane onto a two-dimensional plane in space. It is of course also possible to dispense with a calibration if the relative movement can be ascertained by an image analysis or using a separate measurement system,”).
Regarding claim 15, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein, using the calibration data, the transforming of the raw overview image for calculating the overview image occurs so that image directions in the overview image coincide with - an orientation of selectable directional commands for an adjustable sample stage, wherein the directional commands can be entered via software or a control element on the microscope, or image directions in sample images captured by a sample camera different from the overview camera or image directions visible using a microscope eyepiece (Ohrt: 0014-0015: “A calculation algorithm that is used to calculate an overview contrast image from the at least two overview raw images is chosen in dependence on the type of the illumination and information that is to be extracted from the overview contrast image…an image evaluation calculation algorithm is chosen that is used to extract the information from the overview contrast image. Said information can then be used for example by the user on the screen of a connected computer to initiate further steps as part of the observation and analysis, for example to navigate on the sample, which is accomplished by the image being represented on a screen and the user choosing the sample region of interest for example by way of a mouse click. On account of the image evaluation, the microscope system can then be automatically adjusted to that position.”; Wherein the raw overview images are transformed into a contrast image oriented to coincide with the directional inputs of a user).
As per claim(s) 16, arguments made in rejecting claim(s) 1 are analogous. In addition, Figure 1 and Paragraphs 0059-0060 of Ohrt disclose: A microscopy system including a microscope comprising an overview camera, a sample camera, and at least one computing device.
As per claim(s) 17, arguments made in rejecting claim(s) 1 are analogous. In addition, Paragraph 0060 of Ohrt discloses the integration of the components as either hardware or software in a PC, which implies “commands stored on a non-transitory computer-readable medium”.
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over Ohrt in view of Dasgupta and Wei, and further in view of Amthor et al. (US2020200531A1) hereinafter referenced as Amthor.
Regarding claim 2, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1, wherein the calibration data define or help to determine how raw overview images are transformed into a top view overview image. (Ohrt: Figures 11-12; 0014: “A calculation algorithm that is used to calculate an overview contrast image from the at least two overview raw images is chosen in dependence on the type of the illumination and information that is to be extracted from the overview contrast image.”)
Ohrt in view of Dasgupta and Wei does not disclose expressly: wherein the calibration data define or help to determine how an oblique view is converted into a top view through the transforming of the raw overview image and whether an image mirroring is required for calculating the overview image.
Amthor discloses: an overview camera which can be positioned obliquely from the top with the usage of a mirror, allowing for the calibration of an oblique view into a top view (Amthor: Figure 3; 0021: “the overview camera could thus also be directed obliquely from below at the sample carrier without the ML-based image analysis system failing. Additionally, it is also possible—with certain limitations—to position the overview camera above the sample carrier (i.e., the sample stage). Furthermore, mirrors can be used to reflect one of the sides of the sample carrier towards the overview camera.”; Wherein the image mirroring can be determined if a mirror was used for the creation of the overview image).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to substitute the overview camera disclosed in Ohrt in view of Dasgupta and Wei with the overview camera and mirrors taught in Amthor. The suggestion/motivation for doing so would have been “This way offers greater design freedom in terms of the positioning of the overview camera.” (Amthor: 0039). Further, one skilled in the art could have substituted the elements as described above by known methods with no change in their respective functions, and the substitution would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ohrt in view of Dasgupta and Wei with Amthor to obtain the invention as specified in claim 2.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Ohrt in view of Dasgupta and Wei, and further in view of Kieu et al. (Local blur correction for document images) hereinafter referenced as Kieu.
Regarding claim 6, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1.
Ohrt in view of Dasgupta and Wei does not disclose expressly: wherein one or more of the following processes are carried out: checking whether the text satisfies a minimum size, wherein the text is enlarged relative to surrounding image content in cases where the minimum size is not satisfied; checking whether a text sharpness, a brightness, a contrast or a color tone in the text region meets predetermined criteria, wherein the text sharpness, brightness, contrast or color tone in the text region is modified in cases where the criteria are not met; checking whether there is a partial concealment of text, in which case characters are added to the text with a machine-trained model; checking whether there is smearing of text, in which case an image processing for removal of smearing occurs.
Kieu discloses: checking whether there is smearing of text, in which case an image processing for removal of smearing occurs (Kieu: Abstract: “Therefore, we propose to detect blur and to attenuate its effect on an OCR result. As blur is nonuniform on the document area, we propose a local approach. No prior model is chosen for blur that may be of various natures. Thanks to a local clustering based on a novel blur feature, we build a debluring adapted to the heterogeneous blur. As a result, the blurred image is locally corrected according to the blur type”; Wherein blurring constitutes smearing).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the local deblurring algorithm disclosed by Kieu to deblur blurry text present in Ohrt in view of Dasgupta and Wei. The suggestion/motivation for doing so would have been “The experimental results show a significant improvement of 11% of OCR accuracy” (Kieu: Page 4064; Wherein correcting blur improves OCR accuracy). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ohrt in view of Dasgupta and Wei with Kieu to obtain the invention as specified in claim 6.
Claim(s) 12 is rejected under 35 U.S.C. 103 as being unpatentable over Ohrt in view of Dasgupta and Wei, and further in view of Marda et al. (US2019019022A1) hereinafter referenced as Marda.
Regarding claim 12, Ohrt in view of Dasgupta and Wei discloses: The method according to claim 1.
Ohrt in view of Dasgupta and Wei does not disclose expressly: wherein a transition is generated for a boundary between the transformed text region and an adjacent image content of the overview image by inputting the overview image with the transformed text region into a model trained for image reconstruction, which modifies an image area around the boundary.
Marda discloses: wherein a transition is generated for a boundary between the transformed text region and an adjacent image content of the overview image by inputting the overview image with the transformed text region into a model trained for image reconstruction, which modifies an image area around the boundary (Marda: Figures 2c and 2d; 0069: “FIG. 2D illustrates, on the client device 200 a (though the client device could be client device 200 from FIG. 2A) an updated electronic document 204 a after applying the annotations from the shared state mapping (e.g., based on annotations identified in the physical document 206 ).”; 0071: “the document management system can insert the differences into the electronic document 204 a as images, as described above. In one or more embodiments, the document management system inserts such differences as drawn/depicted. In alternative embodiments, the document management system processes the differences using, for example, by smoothing edges,”; Wherein the algorithm used for smoothing the edges of the inserted differences constitutes a model trained for image reconstruction).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the algorithm used for smoothing the edges of inserted differences taught by Marda for the replacing of distorted text regions with the corrected text regions disclosed by Ohrt in view of Dasgupta and Wei. The suggestion/motivation for doing so would have been to incorporate the corrected text regions into a text without displaying the explicit annotations or mark ups (Marda: Figures 2c-2d; 0017;). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Ohrt in view of Dasgupta and Wei with Marda to obtain the invention as specified in claim 12.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/
Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights