Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Submission dated 06/22/2026 amends claims 1, 7 and 11 and cancels claims 5, 10 and 15. Claims 1-4, 6-9 and 11-16 are pending.
Response to Arguments
Applicant's arguments filed with the submission dated 06/22/2026 have been fully considered but they are not persuasive.
On pages 9-17, the applicant argues that Baheti in view of Bako does teach the following limitations, which are incorporated from the cancelled claims, e.g., claims 5, 10, and 15:
“wherein the segmenting is based on k-means clustering, and
wherein the segmenting comprises:
classifying each pixel associated with the at least one visual text region into a predefined category of one or more predefined categories based on the pixel intensity and color;
generating one or more clusters corresponding to the one or more predefined categories; and
identifying, from the one or more clusters, a cluster corresponding to text characters distinct from clusters corresponding to background.”
On pages 11-12, the applicant argues that “Baheti do not read on the amended independent claim language, which now requires pixel clustering and cluster identification within a visual text region” (see the second full paragraph on page 12 of the submission).
The examiner did not rely on Baheti to teach the above limitations. While the rejection sis based on the combined teaching of Baheti and Bako, when the above limitations were part of the cancelled claims, the examiner relied on the teaching of Bako to teach them. As such, the examiner finds the applicant’s above argument moot.
On pages 12-13, the applicant argues that Bako’s clustering is different from that of the claimed invention because Bako has a different purpose and Bako does not extracts or isolate text pixels as a final output. The examiner disagrees because the clustering’s purpose and whether it extracts or isolates text pixels as a final output are not recited in the claims and hence not relevant.
On page 13, the applicant argues that Bako does not teach performing selection of a text-character cluster for segmentation and extraction from the image and applying clustering after detecting a visual text region. Again, the examiner disagrees because the claim language does not require such a selecting and the specific timing of the clustering.
On pages 13-14, the applicant argues that Bako does not teach the segmentation based on k-means clustering because k-means in Bako is merely an initialization technique. Again, the examiner disagrees because the claim language merely the segmentation to be based on k-means clustering. The claim does recite that the segmentation starts and ends with k-means clustering.
On pages 14, the applicant argues that Bako does not teach the claimed predefined categories. The examiner disagrees because Bako in, e.g., pars. 31-33 and FIG. 3, teach assigning pixels to different categories based on their different color intensities.
On pages 14-15, the applicant argues that Bako’s clustering does not teach the claimed classification. The examiner disagrees because as shown by 308(b) in FIG. 3, while clustering itself may be sufficient, Bako also teaches labeling the clusters as text and background. Moreover, clustering pixels based on their color intensities reads on classifying pixels because the color intensities correspond to and are part of the pixels under the broadest reasonable interpretation.
On pages 15-16, the applicant argues that the combined teaching of Baheti and Bako fails to teach the claimed invention because there is no teaching or motivation to combine. The examiner disagrees because it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Baheti to perform the clustering as taught by Bako because doing so allow yield predictable results of detecting peaks/clusters corresponding to undesired artifacts and removing the artifacts (see par. 31 of Bako and also MPEP 2143(I)(D)).
For these reasons, the examiner finds the applicant’s argument unpersuasive.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6-9, 11-14, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Us patent application publication no. 2014/0168478 to Baheti et al. (hereinafter Baheti) in view of us patent application publication no. 2017/0309003 to Bako et al. (hereinafter Bako).
For claim 1, Baheti as applied disclose a method of extracting text from a video stream, the method comprising:
determining at least one visual text region in each image of a plurality of images (see, e.g., pars. 38-40 and FIG. 2, which teach identifying one or more text regions from obtained images), wherein the video stream comprises the plurality of images in sequential order (see, e.g., pars. 2, 5, 6, 36, 38, and 82-85 and claims 7, 20 and 26, which teach receiving multiple frames/images, e.g., video frames, in sequence), wherein the at least one visual text region comprises a plurality of text characters (see, e.g., pars. 40-42, 45-46, 51, and 84 and FIG. 2, which teach verifying presence of text in the identified regions), and
wherein determining the at least one visual text region is based on analysis of one of (the examiner notes that to be consistent with pars. 49-50 of the specification under the broadest reasonable interpretation, the following limitations are interpreted disjunctively (see MPEP 2111.01):
lines and curves associated with each of the plurality of text characters of the at least one visual text region;
a blob associated with the plurality of text characters along an axis (see, e.g., pars. 39, 49-50, 61-62 and FIGS. 2, 3A, which teach identifying regions of pixels in the image that are connected to one another and different from surrounding pixels in one or more properties, such as color and/or intensity, e.g., maximally stable extremal regions (MSER));
distribution of the plurality of text characters along a major axis; and
a common attribute associated with the plurality of text characters;
segmenting the plurality of text characters within the at least one visual text region from a background associated with the at least one visual text region (see, e.g., pars. 38-40, 57-58 and 65-66 and FIGS. 2, 3A and 4A, which teach extracting one or more text regions from the obtained images); and
upon segmenting, inpainting each of the plurality of text characters with a predefined color (see, e.g., pars. 55, 63 and 65, which teach binarizing the extracted region, and pars. 76-77 and FIGS. 5A, 6A-B, which teach, upon extracting the regions, performing text image enhancement, e.g., deblurring and improving contrast of the region by changing intensities of pixels in the region and correcting over/under exposure of region).
While Baheti as applied does not explicitly teach, Bako in the analogous art teaches:
wherein the segmenting is based on k-means clustering (see, e.g., pars. 30-31 of Bako, which segmenting pixels of text from those of a background), and wherein the segmenting comprises:
classifying each pixel associated with the at least one visual text region into a predefined category of one or more predefined categories based on the pixel intensity and color (see, e.g., pars. 29-32 and FIG. 3 of Bako, which assigning higher color intensity values represent background compared to the values pixel representing text);
generating one or more clusters corresponding to the one or more predefined categories (see, e.g., pars. 29-32 and FIG. 3 of Bako, which generating a color intensity histogram with peaks corresponding to color intensities of pixels representing text and those representing a background); and
identifying, from the one or more clusters, a cluster corresponding to text characters distinct from clusters corresponding to background (see, e.g., pars. 29-32 and FIG. 3 of Bako, which identifying cluster/peak of pixels representing text and cluster/peak representing a background).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Baheti to perform the clustering as taught by Bako because doing so allow yield predictable results of detecting peaks/clusters corresponding to undesired artifacts and removing the artifacts (see par. 31 of Bako and also MPEP 2143(I)(D)).
For claim 7, Baheti as applied teaches a system for extracting text from a video stream (see, e.g., FIGS. 2, 8A and 9), the system comprising:
a processor (see, e.g., pars. 36-37 and FIG. 9); and
a memory storing a plurality of instructions (see, e.g., FIGS. 2 and 9), wherein the plurality of instructions, upon execution by the processor, cause the processor to:
determine at least one visual text region in each image of a plurality of images (see, e.g., pars. 38-40 and FIG. 2, which teach identifying one or more text regions from obtained images), wherein the video stream comprises the plurality of images in sequential order (see, e.g., pars. 2, 5, 6, 36, 38, and 82-85 and claims 7, 20 and 26, which teach receiving multiple frames/images, e.g., video frames, in sequence), wherein the at least one visual text region comprises a plurality of text characters (see, e.g., pars. 40-42, 45-46, 51, and 84 and FIG. 2, which teach verifying presence of text in the identified regions), and
wherein determining the at least one visual text region is based on analysis of one of (the examiner notes that to be consistent with pars. 49-50 of the specification under the broadest reasonable interpretation, the following limitations are interpreted disjunctively (see MPEP 2111.01):
lines and curves associated with each of the plurality of text characters of the at least one visual text region;
a blob associated with the plurality of text characters along an axis (see, e.g., pars. 39, 49-50 and 61-62 and FIGS. 2 and 3A, which teach identifying regions of pixels in the image that are connected to one another and different from surrounding pixels in one or more properties, such as color and/or intensity, e.g., maximally stable extremal regions (MSER));
distribution of the plurality of text characters along a major axis; and
a common attribute associated with the plurality of text characters;
segment the plurality of text characters within the at least one visual text region from a background associated with the at least one visual text region (see, e.g., pars. 38-40, 57-58 and 65-66 and FIGS. 2, 3A and 4A, which teach extracting one or more text regions from the obtained images); and
upon segmenting, inpaint each of the plurality of text characters with a predefined color (see, e.g., pars. 55, 63 and 65, which teach binarizing the extracted region, and pars. 76-77 and FIGS. 5A and 6A-B, which teach, upon extracting the regions, performing text image enhancement, e.g., deblurring and improving contrast of the region by changing intensities of pixels in the region and correcting over/under exposure of region).
While Baheti as applied does not explicitly teach, Bako in the analogous art teaches:
wherein the segmenting is based on k-means clustering (see, e.g., pars. 30-31 of Bako, which segmenting pixels of text from those of a background), and wherein the segmenting comprises:
classifying each pixel associated with the at least one visual text region into a predefined category of one or more predefined categories based on the pixel intensity and color (see, e.g., pars. 29-32 and FIG. 3 of Bako, which assigning higher color intensity values represent background compared to the values pixel representing text);
generating one or more clusters corresponding to the one or more predefined categories (see, e.g., pars. 29-32 and FIG. 3 of Bako, which generating a color intensity histogram with peaks corresponding to color intensities of pixels representing text and those representing a background); and
identifying, from the one or more clusters, a cluster corresponding to text characters distinct from clusters corresponding to background (see, e.g., pars. 29-32 of Bako and FIG. 3, which identifying cluster/peak of pixels representing text and cluster/peak representing a background).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Baheti to perform the clustering as taught by Bako because doing so allow yield predictable results of detecting peaks/clusters corresponding to undesired artifacts and removing the artifacts (see par. 31 of Bako and also MPEP 2143(I)(D)).
For claim 11, Baheti as teaches discloses a non-transitory computer-readable medium storing computer-executable instructions for extracting text from a video stream (see, e.g., pars. 104-107 and FIG. 9), the computer-executable instructions configured for:
determining at least one visual text region in each image of a plurality of images (see, e.g., pars. 38-40 and FIG. 2, which teach identifying one or more text regions from obtained images), wherein the video stream comprises the plurality of images in sequential order (see, e.g., pars. 2, 5, 6, 36, 38, and 82-85 and claims 7, 20 and 26, which teach receiving multiple frames/images, e.g., video frames, in sequence), wherein the at least one visual text region comprises a plurality of text characters (see, e.g., pars. 40-42, 45-46, 51, and 84 and FIG. 2, which teach verifying presence of text in the identified regions), and
wherein determining the at least one visual text region is based on analysis of one of (the examiner notes that to be consistent with pars. 49-50 of the specification under the broadest reasonable interpretation, the following limitations are interpreted disjunctively (see MPEP 2111.01):
lines and curves associated with each of the plurality of text characters of the at least one visual text region;
a blob associated with the plurality of text characters along an axis (see, e.g., pars. 39, 49-50 and 61-62 and FIGS. 2, 3A, which teach identifying regions of pixels in the image that are connected to one another and different from surrounding pixels in one or more properties, such as color and/or intensity, e.g., maximally stable extremal regions (MSER));
distribution of the plurality of text characters along a major axis; and
a common attribute associated with the plurality of text characters;
segmenting the plurality of text characters within the at least one visual text region from a background associated with the at least one visual text region (see, e.g., pars. 38-40, 57-58 and 65-66 and FIGS. 2, 3A and 4A, which teach extracting one or more text regions from the obtained images); and
upon segmenting, inpainting each of the plurality of text characters with a predefined color (see, e.g., pars. 55, 63 and 65, which teach binarizing the extracted region, and pars. 76-77 and FIGS. 5A and 6A-B, which teach, upon extracting the regions, performing text image enhancement, e.g., deblurring and improving contrast of the region by changing intensities of pixels in the region and correcting over/under exposure of region).
While Baheti as applied does not explicitly teach, Bako in the analogous art teaches:
wherein the segmenting is based on k-means clustering (see, e.g., pars. 30-31 of Bako, which segmenting pixels of text from those of a background), and wherein the segmenting comprises:
classifying each pixel associated with the at least one visual text region into a predefined category of one or more predefined categories based on the pixel intensity and color (see, e.g., pars. 29-32 and FIG. 3 of Bako, which assigning higher color intensity values represent background compared to the values pixel representing text);
generating one or more clusters corresponding to the one or more predefined categories (see, e.g., pars. 29-32 and FIG. 3 of Bako, which generating a color intensity histogram with peaks corresponding to color intensities of pixels representing text and those representing a background); and
identifying, from the one or more clusters, a cluster corresponding to text characters distinct from clusters corresponding to background (see, e.g., pars. 29-32 and FIG. 3 of Bako, which identifying cluster/peak of pixels representing text and cluster/peak representing a background).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Baheti to perform the clustering as taught by Bako because doing so allow yield predictable results of detecting peaks/clusters corresponding to undesired artifacts and removing the artifacts (see par. 31 of Bako and also MPEP 2143(I)(D)).
For claims 2, 8 and 12, Baheti in view of Bako teaches that the analysis of lines and curves comprises:
detecting one or more corners formed with intersection of the lines and curves associated with each of the plurality of text characters of the at least one visual text region (see, e.g., pars. 40, 52-54 and 65-66 and FIGS. 3B-C of Baheti, which teach computing stroke width by detecting a number of points corresponding to intersection of line and curves in a character);
determining pixel intensity and average axial distance between the one or more corners (see, e.g., pars. 40, 52-54 and 65-66 and FIGS. 3B-C of Baheti, which teach computing a variance of the stroke widths by averaging the stroke width, which are defined by pixel intensities); and
determining the at least one visual text region based on the pixel intensity and the average axial distance (see, e.g., pars. 40, 52-54 and 65-66 and FIGS. 3B-C of Baheti, which teach classifying the region as text based on the variance of the stroke widths).
For claims 3 and 13, Baheti in view of Bako teaches that the blob associated with the plurality of text characters along an axis is analyzed based on maximally stable extremal visual text regions (MSER) model (see, e.g., pars. 39, 49-50 and 61-62 and FIGS. 2 and 3A of Baheti, which teach identifying regions of pixels in the image that are connected to one another and different from surrounding pixels in one or more properties, such as color and/or intensity, e.g., maximally stable extremal regions (MSER)).
For claims 4 and 14, Baheti in view of Bako teaches that the common attribute is one of a font, a size, a color, or a spacing associated with each of the plurality of text characters (see, e.g., pars. 65-66 of Baheti, which teach determining whether a line of pixels of a common binary value is present in the region, and pars. 41, 55 and 75 and FIGS. 2 and 5A-B, which teach determining text image quality of the regions by checking whether an attribute of the region meets an OCR threshold, wherein the attribute is a text/font size).
For claim 6 and 16, Baheti in view of Bako teaches that each of the plurality of text characters is inpainted with Black color (see, e.g., pars. 55, 63 and 65 of Baheti, which teach binarizing the extracted region, and pars. 76 and 77 and FIGS. 5A and 6A-B, which teach, upon extracting the regions, performing text image enhancement, e.g., deblurring and improving contrast of the region by changing intensities of pixels in the region and correcting over/under exposure of region).
For claim 9, Baheti in view of Bako teaches that the blob associated with the plurality of text characters along an axis is analyzed based on maximally stable extremal visual text regions (MSER) model (see, e.g., pars. 39, 49-50 and 61-62 and FIGS. 2 and 3A of Baheti, which teach identifying regions of pixels in the image that are connected to one another and different from surrounding pixels in one or more properties, such as color and/or intensity, e.g., maximally stable extremal regions (MSER)); and
wherein the common attribute is one of a font, a size, or a spacing associated with each of the plurality of text characters (see, e.g., pars. 65-66 of Baheti, which teach determining whether a line of pixels of a common binary value is present in the region, and pars. 41, 55 and 75 and FIGS. 2 and 5A-B, which teach determining text image quality of the regions by checking whether an attribute of the region meets an OCR threshold, wherein the attribute is a text/font size).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WOO RHIM whose telephone number is (571)272-6560. The examiner can normally be reached Mon - Fri 9:30 am - 6:00 pm et.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WOO C RHIM/Examiner, Art Unit 2676
