CTNF 18/723,870 CTNF 101555 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. 07-30-06 This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: obtaining module, conversion module, identification module, optimization unit, merging module, and classification module in claims 14-17, 19-21, and 23. Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-8, 10-11, 14-21, and 23-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The method of claim 1 is directed to a process, which is one of the statutory categories of invention, and passes Step 1: Statutory Category- MPEP § 2106.03. However, the following elements of Claim 1 recite steps that can be performed in the human mind or with pen and paper, therefore failing Step 2A Prong One. These steps constitute mental processes because they describe acts of observation, evaluation, and judgement that a human can perform mentally. For example, a human can mentally identify or visually select a region of an image that contains text, mentally interpret a line as defining a region of interest and use that region to identify text, and read text within a selected region and write it down. obtaining a target text area in a first image frame, wherein the target text area is a to-be- identified text area; converting a first drawn line in the target text area into a first detection area, wherein the first detection area is used to identify a text area marked by the first drawn line; and identifying a text area in the first detection area to obtain a user note. Claim 1 fails Step 2A Prong Two because the additional elements beyond the judicial exception, including obtaining an image frame, do not integrate the judicial exception into a practical application. Obtaining an image frame is insignificant extra-solution activity (MPEP § 2106.05(g)), and the method does not improve the functioning of a computer or any other technology or technical field (MPEP § 2106.05(a)) as it merely applies the abstract idea on a computer (MPEP § 2106.05(f)). The claim also does not impose meaningful limits on the computer components such that the method is tied to a particular machine; the method may operate on any generic computing system (MPEP § 2106.05(b)). Claim 1 also fails Step 2B, as these generic elements are well-understood, routine, and conventional (WURC), adding nothing significantly more than the abstract idea itself (MPEP § 2106.07(a)(III)). Obtaining an image frame is WURC (see MPEP § 2106.05(d)). Accordingly, claim 1 is rejected. Claim 14 contains this identical ineligible subject matter, with the only additional elements beyond the judicial exception being a first image frame originating from media information, an obtaining module, a conversion module, and an identification module. These additional elements do not integrate the judicial exception into a practical application (see claim 1 analysis above) and are WURC (see MPEP § 2106.05(d)). Therefore, claim 14 is rejected. Claims 2-11 fail Step 2A Prong One as they further recite mental processes. In claim 2, a human can draw or imagine multiple rectangles over a line, mentally “stack” or arrange them, compare overlaps, select the one with the largest overlapping degree, expand a line into a region, and define a bounding area. In claim 3, a human can partition a region mentally or on paper, estimate a proportion, compare to a threshold, decide to remove, and combine remaining regions into a final area. In claim 4, a human can compare two frames, detect differences, and decide which region is the target. In claim 5, a human can count a quantity of text areas and visually measure size, angle, and locations of text areas. In claim 6, a human can compare a quantity of text areas in the first image frame with a quantity of text areas in the second image frame, observe changes and identify a new object, determine the new text area as the target text area, associate text with a person, interpret “importance” or size, and compare semantic relevance. In claim 7, a human can estimate distance, compare to a threshold, obtain two user notes (reading and writing), decide closeness, and merge regions. In claim 8, a human can combine notes into one and recognize paragraph structure. In claim 10, a human can check membership in a set, organize notes into groups, create a new category, and assign a note. In claim 11, a human can highlight text with color and decide formatting choices. These claims fail Step 2A Prong Two and Step 2B as there are no additional elements beyond the judicial exception. Claims 14-21 and 23-24 also contain this identical ineligible subject matter, with the only additional elements beyond the judicial exception being an optimization unit, merging module, and classification module. These additional elements do not integrate the judicial exception into a practical application (see claim 1 analysis above) and are WURC (see MPEP § 2106.05(d)). Therefore, claims 14-21 and 23-24 are also rejected. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15 AIA Claim s 1-3, 5, 7-8, 10-11, 14-16, and 23-24 are rejected under 35 U.S.C. 102 ( a)(1 ) as being anticipated by Miura (JP7767764B2) . Regarding Claim 1, Miura teaches a note generating method, wherein the method comprises: Paragraph [0001]: “The present invention relates to a document processing apparatus, a document processing method, and a program capable of character recognition processing of documents such as forms .” obtaining a target text area in a first image frame, wherein the target text area is a to-be- identified text area; Paragraph [0005]: “Layout analysis--since the layout changes depending on the document, the text area, ruled lines, and image area of the document are analyzed and divided .” Paragraph [0012]: “a detection means for detecting an object indicating the presence of an annotation outside the standard area ” Explanation: The reference discloses identifying predefined and non-pre-defined (to-be-identified) text areas via layout/OCR processing. The “target text area…to-be-identified” corresponds to annotation areas outside the standard areas, which must be detected/identified. converting a first drawn line in the target text area into a first detection area, wherein the first detection area is used to identify a text area marked by the first drawn line; and Paragraph [0012]: “the detection means sets the area where the object is detected as an attention area, expands the detection area around the attention area to detect the object , and repeats expanding and detecting the detection area until no objects are detected in any of the expanded detection areas , and the specification means specifies the annotation area based on the detection result of the detection means … a document processing device according to any one of the preceding paragraphs 1 to 5, wherein, when the detection means detects a leader line as one of the objects, it expands the detection area in the direction in which the leader line extends to detect the end of the leader line, and the specification means specifies an annotation area near the end of the leader line .” Paragraph [0051]: “when an indication line is detected as one of the objects in the nearby area, the detection area is expanded in the direction in which the indication line extends, the end of the indication line is detected, and the annotation area is identified near the end of the indication line .” Paragraph [0053]: “On the other hand, if the instruction line 10 is detected in step S05 (YES in step S05), an annotation is often written near the end of the instruction line 10 . Therefore, the instruction line is traced to its end in step S08, and then the annotation area where the annotation is written is identified in step S04 .” Explanation: “First drawn line” corresponds to indication line/instruction line/leader line. “Convert into detection area” corresponds to setting attention area and expanding detection region. “Identify text area marked by the line” corresponds to tracing to annotation area near line end (e.g., leader line and indication line). identifying a text area in the first detection area to obtain a user note. Paragraph [0043]: “ The annotation region identifying unit 242c identifies a region where an annotation exists, that is, an annotation region.” Paragraph [0044]: “The character recognition unit 242b performs OCR processing on the identified annotation region and extracts the annotation as characters . Note that instead of extracting characters by OCR processing, the identified annotation region may be extracted as an image.” Paragraph [0052]: “If characters are detected in step S03 (YES in step S03), an annotation area in which characters are written is identified in step S04 because an annotation may be written outside the vicinity area 7 . The annotation may be extracted by extracting character information by performing OCR processing on the annotation area, or by extracting an image of the annotation area . Explanation: “Identifying text area” corresponds to annotation area identification (unit 242c). “User note” corresponds to annotation (handwritten or printed). Regarding Claim 2, Miura teaches the method according to claim 1, wherein the converting a first drawn line in the target text area into a first detection area comprises: creating a plurality of first rectangles that overlap with the first drawn line in the target text area, wherein the plurality of first rectangles are sequentially stacked; Paragraph [0058]: “First, a region in which an object is detected in the neighborhood region 7 is set as a region of interest , and the detection region is expanded from the coordinates of the region of interest to surrounding pixels .” Paragraph [0059]: “In this way, expansion of the detection area and object detection are repeated until no object is detected in the entire detection area.” Paragraph [0060]: “As another process for identifying the annotation area 9, a rectangle may be cut out along a predetermined size and direction based on the position where the character is detected, and this cut-out rectangle may be identified as the annotation area 9 , and an annotation extraction process may be performed on this identified annotation area 9.” Explanation: Sequential expansion regions correspond to stacked/overlapping rectangular regions. Explicit rectangle embodiment confirms geometric regions construction. creating a second drawn line in a first rectangle with a largest overlapping degree, wherein the second drawn line is parallel to a long side of the first rectangle with the largest overlapping degree; and Paragraph [0012]: “A document processing device according to any one of the preceding paragraphs 1 to 5, wherein, when the detection means detects a leader line as one of the objects, it expands the detection area in the direction in which the leader line extends to detect the end of the leader line, and the specification means specifies an annotation area near the end of the leader line .” Paragraph [0053]: “On the other hand, if the instruction line 10 is detected in step S05 (YES in step S05), an annotation is often written near the end of the instruction line 10 . Therefore, the instruction line is traced to its end in step S08, and then the annotation area where the annotation is written is identified in step S04 .” Paragraph [0063]: “A Hough transform is used to detect line segments , and a pattern matching technique for feature quantities is used to detect arrows.” Explanation: Selecting the dominant/continuing line corresponds to largest overlap/strongest detected line segment. creating a second rectangle based on the second drawn line, wherein the second rectangle is used as the first detection area, the second drawn line is located in the second rectangle, the second drawn line is parallel to a long side of the second rectangle, and a length of a short side of the second rectangle is greater than a row height of the target text area. Paragraph [0012]: “the detection means sets the area where the object is detected as an attention area, expands the detection area around the attention area to detect the object , and repeats expanding and detecting the detection area until no objects are detected in any of the expanded detection areas , and the specification means specifies the annotation area based on the detection result of the detection means” Paragraph [0059]: “At the point when no objects are detected, the series of expanded areas are identified as an annotation area 9 , as shown by the dotted lines in Fig. 7(C), and an annotation extraction process is performed on this identified annotation area 9.” Paragraph [0060]: “As another process for identifying the annotation area 9, a rectangle may be cut out along a predetermined size and direction based on the position where the character is detected, and this cut-out rectangle may be identified as the annotation area 9 , and an annotation extraction process may be performed on this identified annotation area 9.” Paragraph [0061]: “For example, when part of the characters of the annotation 51 is detected in the vicinity region 7 as shown by the circle in Fig. 8A, a rectangle that is long in the horizontal direction from that position may be cut out as shown in Fig. 8B to identify the annotation region 9 , and annotation extraction processing may be performed on this identified annotation region 9.” Explanation: Rectangle derived from detected line corresponds to annotation/detection area. This directly corresponds to “second rectangle = detection area.” Horizontal rectangle aligned with text lines corresponds to being parallel to long side. Dimensioning relative to text corresponds to the short side being greater than the row height. Regarding Claim 3, Miura teaches the method according to claim 2, wherein after the creating a second rectangle based on the second drawn line, the method further comprises: dividing the second rectangle into a plurality of sub-rectangles; and Paragraph [0058]: “First, a region in which an object is detected in the neighborhood region 7 is set as a region of interest , and the detection region is expanded from the coordinates of the region of interest to surrounding pixels . For example, as shown in Fig. 7A, the pixel indicated by the thick frame in which the object is first detected is set as the region of interest 8, and as shown in Fig. 7B, object detection is performed on the surrounding pixels of the region of interest 8 . For a pixel where an object has been detected , that pixel is set as a region of interest 8, and object detection is performed on the surrounding pixels .” Paragraph [0059]: “In this way, expansion of the detection area and object detection are repeated until no object is detected in the entire detection area. ” Explanation: Expansion into surrounding pixels corresponds to subdivision into smaller regions (sub-rectangles/pixels). Iterative processing divides the detection region into multiple subregions. removing, from the plurality of sub-rectangles, a sub-rectangle whose pixel proportion is less than a preset first threshold, and using a third rectangle formed by remaining sub-rectangles as the first detection area. Paragraph [0051]: “If an indication line 10 has not been detected (NO in step S05), it is checked in step S06 whether a predetermined threshold value defining the vicinity region 7 has been reached , and if not (NO in step S06), the detection region is expanded in step S07, and then the process returns to step S02, and the detection determination of a character or indication line is repeated until the threshold value is reached .” Paragraph [0059]: “In this way, expansion of the detection area and object detection are repeated until no object is detected in the entire detection area. At the point when no objects are detected, the series of expanded areas are identified as an annotation area 9 , as shown by the dotted lines in Fig. 7(C), and an annotation extraction process is performed on this identified annotation area 9.” Paragraph [0060]: “As another process for identifying the annotation area 9, a rectangle may be cut out along a predetermined size and direction based on the position where the character is detected, and this cut-out rectangle may be identified as the annotation area 9 , and an annotation extraction process may be performed on this identified annotation area 9.” Explanation: The reference uses threshold-based filtering. Regions without detected objects are effectively removed, and detection continues until threshold condition is satisfied. This corresponds to removing low-signal (low pixel proportion) regions. The remaining detected regions are merged into the final annotation area (third rectangle), which directly corresponds to forming the final detection area. Regarding Claim 7, Miura teaches the method according to claim 1,wherein a second detection area further exists in the target text area, the second detection area is obtained by converting a third drawn line in a third image frame, the third image frame is previous to the first image frame, and a plurality of image frames exist between the third image frame and the first image frame; and the identifying a text area in the first detection area to obtain a user note comprises: Paragraph [0013]: “When an object is detected, an annotation area in which the annotation is written is identified, and the annotation is extracted from the identified annotation area .” Paragraph [0043]: “ The annotation area specifying unit 242c specifies an area in which an annotation exists, that is, an annotation area .” Paragraph [0044]: “The character recognition unit 242b performs OCR processing on the specified annotation area and extracts the annotation as characters .” Paragraph [0051]: “when an indication line is detected as one of the objects in the nearby area, the detection area is expanded in the direction in which the indication line extends, the end of the indication line is detected, and the annotation area is identified near the end of the indication line .” Paragraph [0053]: “On the other hand, if the instruction line 10 is detected in step S05 (YES in step S05), an annotation is often written near the end of the instruction line 10 . Therefore, the instruction line is traced to its end in step S08, and then the annotation area where the annotation is written is identified in step S04 .” Paragraph [0059]: “In this way, expansion of the detection area and object detection are repeated until no object is detected in the entire detection area . At the point when no objects are detected, the series of expanded areas are identified as an annotation area 9 , as shown by the dotted lines in Fig. 7(C), and an annotation extraction process is performed on this identified annotation area 9.” Explanation: These disclosures show multiple detection/annotation areas (first and second detection areas). The drawn line [Wingdings font/0xE0] traced [Wingdings font/0xE0] detection area at endpoint pipeline corresponds to converting a drawn line into a detection area. The reference also teaches multi-step detection across regions, which corresponds functionally to multiple frames. Lastly, the reference teaches identifying text areas and extracting user notes. when a distance between the text area in the first detection area and a text area in the second detection area is greater than or equal to a preset second threshold, respectively identifying the text area in the first detection area and the text area in the second detection area to obtain two user notes; Paragraph [0066]: “In this case, the annotations may be displayed as individual images as shown in Fig. 11, or as an image of the entire document including the annotations, or as character information as shown in Fig. 12 .” Paragraph [0067]: “On the display screen for confirming the processing results, annotations are generally written near the related items in the standard area 6, or if they cannot be written, they are written in a different location by drawing an indication line 10 .” Explanation: Supports multiple independent annotations (separate notes) or when a distance between the text area in the first detection area and a text area in the second detection area is less than a preset second threshold, merging the first detection area and the second detection area into a third detection area, and identifying a text area in the third detection area to obtain the user note. Paragraph [0059]: “In this way, expansion of the detection area and object detection are repeated until no object is detected in the entire detection area . At the point when no objects are detected, the series of expanded areas are identified as an annotation area 9 , as shown by the dotted lines in Fig. 7(C), and an annotation extraction process is performed on this identified annotation area 9.” Explanation: Expansion merges nearby regions into a single annotation area, corresponding to merging detection areas. Regarding Claim 8, Miura teaches the method according to claim 7, wherein after the respectively identifying the text area in the first detection area and the text area in the second detection area to obtain two user notes, the method further comprises: merging the two user notes to obtain a new user note, wherein the two user notes are located in a same paragraph, and the new user note comprises other texts other than the two user notes in the paragraph and the two user notes that are highlighted. Paragraph [0012]: “The document processing device according to any one of paragraphs 1 to 3, wherein the extraction means extracts the annotation as an image .” Paragraph [0047]: “ Annotation 52, "Changed to 200 sheets of copy paper," is written near the end of instruction line 10 drawn from the corresponding item in the standard area , "500 sheets of copy paper.” Paragraph [0050]: “For comparison, in the conventional example shown in FIG. 6B, annotation extraction processing is performed on all areas 71 outside the standard area 6 .” Paragraph [0059]: “At the point when no objects are detected, the series of expanded areas are identified as an annotation area 9 , as shown by the dotted lines in Fig. 7(C), and an annotation extraction process is performed on this identified annotation area 9.” Paragraph [0060]: “As another process for identifying the annotation area 9, a rectangle may be cut out along a predetermined size and direction based on the position where the character is detected, and this cut-out rectangle may be identified as the annotation area 9 , and an annotation extraction process may be performed on this identified annotation area 9.” Paragraph [0067]: “On the display screen for confirming the processing results, annotations are generally written near the related items in the standard area 6, or if they cannot be written, they are written in a different location by drawing an indication line 10 .” Explanation: The reference teaches combining multiple detected regions into a unified extracted annotation. These disclosures also correspond to spatial grouping (same paragraph/region context). Lastly, extraction of the entire region includes annotation text and surrounding text/content. Regarding Claim 10, Miura teaches the method according to claim 1, wherein a target symbol exists in the target text area, and after the identifying a text area in the first detection area to obtain a user note, the method further comprises: Paragraph [0012]: “ a detection means for detecting an object indicating the presence of an annotation outside the standard area , a specification means for specifying an annotation area where the annotation is written when the object is detected by the detection means, and an extraction means for extracting the annotation from the annotation area specified by the specification means .” Paragraph [0043]: “ The annotation region identifying unit 242c identifies a region where an annotation exists, that is, an annotation region.” Paragraph [0044]: “The character recognition unit 242b performs OCR processing on the identified annotation region and extracts the annotation as characters . Note that instead of extracting characters by OCR processing, the identified annotation region may be extracted as an image.” Paragraph [0042]: “The object detector 242a detects objects within the vicinity area, such as at least some of the characters of an annotation, or an indication line .” Explanation: Target symbol corresponds to detected objects (characters, symbols, lines) within text/annotation areas. when the target symbol is comprised in a preset symbol set, adding the user note to a user note set corresponding to the target symbol; Paragraph [0012]: “A document processing device according to any one of the preceding paragraphs 1 to 7, wherein the annotation extracted by the extraction means is associated with the most relevant item in the standard area .” Paragraph [0067]: “On the display screen for confirming the processing results, annotations are generally written near the related items in the standard area 6 , or if they cannot be written, they are written in a different location by drawing an indication line 10. For this reason, it is advisable to associate the extracted annotations with the items in the standard area 6, and when displaying the annotations, to make it easier to visually identify which items they are related to when checking the results, so that the associated items and annotations are displayed in correspondence with each other .” Explanation: Association corresponds to grouping notes based on a corresponding element (symbol/item). or when the target symbol is not comprised in the symbol set, adding the target symbol to the symbol set, creating a user note set corresponding to the target symbol, and then adding the user note to the user note set corresponding to the target symbol. Paragraph [0012]: “the detection means sets the area where the object is detected as an attention area, expands the detection area around the attention area to detect the object , and repeats expanding and detecting the detection area until no objects are detected in any of the expanded detection areas…” Paragraph [0059]: “In this way, expansion of the detection area and object detection are repeated until no object is detected in the entire detection area . At the point when no objects are detected, the series of expanded areas are identified as an annotation area 9 , as shown by the dotted lines in Fig. 7(C), and an annotation extraction process is performed on this identified annotation area 9.” Explanation: New detection [Wingdings font/0xE0] new annotation area [Wingdings font/0xE0] effectively creating a new grouping/set. Regarding Claim 11, Miura teaches the method according to claim 1, wherein the first detection area is a first color block, and the first color block is used to cover the text area marked by the first drawn line, and wherein a format of the user note is determined based on an instruction entered by the user, and the format of the user note comprises at least one of the following: a font of the user note, a color of the user note, a thickness of the user note, a location of the user note, and a paragraph identifier of the user note. Paragraph [0012]: “the detection means sets the area where the object is detected as an attention area, expands the detection area around the attention area to detect the object , and repeats expanding and detecting the detection area until no objects are detected in any of the expanded detection areas…” Paragraph [0037]: “The operation unit 25 is configured by, for example, a mouse, a keyboard, etc., and accepts user input for OCR processing , etc. When the results of the OCR processing are displayed on the display unit 21 of the server 2, the operation unit also accepts user input for the display of the results .” Paragraph [0045]: “The recognition result output unit 243 outputs the OCR processing result for the standard region and the annotation extraction processing result for the nearby region to the terminal device 3 or the like . The results may be output and displayed on the display unit 21 of the server 2 itself .” Paragraph [0050]: “ The gray zone shown in FIG. 6A is the nearby area 7.” Paragraph [0051]: “when an indication line is detected as one of the objects in the nearby area, the detection area is expanded in the direction in which the indication line extends, the end of the indication line is detected, and the annotation area is identified near the end of the indication line .” Paragraph [0053]: “On the other hand, if the instruction line 10 is detected in step S05 (YES in step S05), an annotation is often written near the end of the instruction line 10 . Therefore, the instruction line is traced to its end in step S08, and then the annotation area where the annotation is written is identified in step S04 .” Paragraph [0059]: “At the point when no objects are detected, the series of expanded areas are identified as an annotation area 9 , as shown by the dotted lines in Fig. 7(C), and an annotation extraction process is performed on this identified annotation area 9.” Paragraph [0066]: “In this case, the annotations may be displayed as individual images as shown in Fig. 11, or as an image of the entire document including the annotations, or as character information as shown in Fig. 12 .” Paragraph [0067]: “For this reason, it is advisable to associate the extracted annotations with the items in the standard area 6, and when displaying the annotations, to make it easier to visually identify which items they are related to when checking the results, so that the associated items and annotations are displayed in correspondence with each other .” Explanation: These disclosures show explicit visual regions (blocks/areas) corresponding to claimed “color block.” The detection area (block) covers the text indicated by drawn line. User input and output formatting supports format determination. These excerpts support the format of the user note, which comprises location (next to item), visual presentation (formatting), and structured display (paragraph/item association). Regarding Claim 14, Miura teaches all of the limitations of claim 1 above because claim 14 recites a note generating apparatus comprising an obtaining module, conversion module, and identification module that perform substantially the same steps as claim 1. Miura teaches a document processing apparatus that contains a display unit 21, storage unit 22, communication unit 23, control unit 24, operation unit 25, standard area recognition unit 241, standard area outside recognition unit 242, outside fixed region recognition unit 242, object detection unit 242a, annotation area specifying unit 242c, character recognition unit 242b, and recognition result output unit 243 that are equivalent to the modules specified above. Miura further teaches that “the first image frame originates from media information” (paragraph [0029] (shown below)). Paragraph [0001]: “The present invention relates to a document processing apparatus , a document processing method, and a program capable of character recognition processing of documents such as forms.” Paragraph [0029]: “The image reading device 1 is a device that reads various standard and non-standard documents , and examples thereof include a multifunction printer, a handheld scanner, a smartphone equipped with a camera function, a personal computer (PC ), etc. The image reading device 1 reads the document, converts it into image data, which is electronic data, and transmits it to the server 2 .” Paragraph [0085]: “REFERENCE SIGNS LIST 1 Image reading device 2 Server 3 Terminal device 4 Network 5 Form 6 Fixed area 7 Nearby area 8 Attention area 9 Annotation area 10 Support line 11 End of support line 11 Light receiving unit 21 Display unit 22 Storage unit 23 Communication unit 24 Control unit 24a CPU 24b ROM 24c RAM 51 to 53 Annotation 55 Item 241 Fixed area recognition unit 242 Outside fixed area recognition unit 242a Object detection unit 242b Character recognition unit 242c Annotation area identification unit 243 Recognition result output unit ” Regarding Claim 15, Miura teaches the apparatus according to claim 14, and additional limitations are met as in the consideration of claim 2 above. Regarding Claim 16, Miura teaches the apparatus according to claim 15, and additional limitations are met as in the consideration of claim 3 above. Miura further teaches the optimization unit (see claim 14 above). Regarding Claim 23, Miura teaches the apparatus according to claim 14, and additional limitations are met as in the consideration of claim 10 above. Miura further teaches the classification module (see claim 14 above). Regarding Claim 24, Miura teaches the apparatus according to claim 14, and additional limitations are met as in the consideration of claim 11 above . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 4, 5, 17, 18, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Miura in view of Parmar and Bunk (US2021056251A1) . Regarding Claim 4, Miura teaches the method according to claim 1, but fails to teach that obtaining a target text area in a first image frame comprises: when state information about text areas in the first image frame is different from state information about text areas in a second image frame, determining the target text area in the first image frame based on the state information about the text areas in the first image frame, wherein the second image frame is an image frame previous to the first image frame. While Miura teaches determining text areas based on state/layout information (positions, structure) (see paragraphs [0005] and [0049]), he does not teach an explicit frame-to-frame comparison. However, Parmar and Bunk explicitly teach multi-frame temporal analysis and comparison of writing/text state. Parmar and Bunk teach frame-based processing, stating that they “identify at least one of key frames and key groups in the writing” (paragraph [0006]), and that “key frames are specially generated images which collectively contain all or nearly all of the written information from the video” (paragraph [0032]). Parmar and Bunk also teach temporal ordering and comparison, stating that they perform “time ordering of the one or more elements of the at least one key frames and key groups” (paragraph [0006]). Importantly, Parmar and Bunk also teach change detection, stating that they “extract time stamped key frames using change detection from the video(s) with the key frames representing all the information written on the boards” (paragraph [0037]). Lastly, Parmar and Bunk teach state information of text, stating that they perform “OCR: Optical Character Recognition…general idea of extracting text from an image, includes subtasks such as detecting lines on a page, detecting words within each line, and converting words into text (e.g., to Unicode)” (paragraph [0030]). Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method of Miura to include comparing state information of text areas across multiple image frames as taught by Parmar and Bunk. Incorporating this teaching into Miura would improve detection of annotations and text regions over time, enhance robustness in dynamic or multi-frame image sequences, and allow for more accurate identification of relevant text areas. This represents the use of a known technique (temporal change detection) to improve similar devices (OCR/layout analysis) in the same way, which would have yielded predictable results. Regarding Claim 5, Miura in view of Parmar and Bunk teaches the method according to claim 4, and Miura further teaches that the state information about the text areas comprises at least one of the following: a quantity of the text areas, sizes of the text areas, angles of the text areas, and locations of the text areas. Paragraph [0005]: “Layout analysis--since the layout changes depending on the document, the text area, ruled lines, and image area of the document are analyzed and divided .” Paragraph [0049]: “The definition data includes information on multiple items in the standard area 6 and the reading position, and OCR processing begins from the reading position, obtaining multiple items such as "orderer," "person in charge," "product number," and "product name" and corresponding character strings .” Paragraph [0060]: “Further, as another process for identifying the annotation area 9, a rectangle is cut out along a predetermined size and direction from the position where the character is detected as a base point , and this cut-out rectangle is identified as the annotation area 9 ., annotation extraction processing may be performed on the specified annotation region 9.” Paragraph [0061]: “For example, when part of the characters of the annotation 51 is detected in the vicinity region 7 as shown by the circle in Fig. 8A, a rectangle that is long in the horizontal direction from that position may be cut out as shown in Fig. 8B to identify the annotation region 9 , and annotation extraction processing may be performed on this identified annotation region 9.” Explanation: Quantity and location = paragraph [0049], size = paragraph [0060], angles = paragraph [0061] Regarding Claim 17, Miura teaches the apparatus according to claim 14, and additional limitations are met as in the consideration of claim 4 above. Regarding Claim 18, Miura in view of Parmar and Bunk teaches the apparatus according to claim 17, and additional limitations are met as in the consideration of claim 5 above. Regarding Claim 20, Miura in view of Parmar and Bunk teaches the apparatus according to claim 17, and additional limitations are met as in the consideration of claim 7 above. Regarding Claim 21, Miura in view of Parmar and Bunk teaches the apparatus according to claim 20, and additional limitations are met as in the consideration of claim 8 above. Miura further teaches the merging module (see claim 14 above) . 07-21-aia AIA Claim s 6 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Miura in view of Parmar and Bunk, further in view of Su and Zhang (US12008761B2) . Regarding Claim 6, Miura in view of Parmar and Bunk teaches the method according to claim 4, and Miura further teaches that determining the target text area in the first image frame based on the state information about the text areas in the first image frame comprises: when a new text area exists in the first image frame, determining the new text area as the target text area; Paragraph [0008]: “In this information processing device, in order to extract annotations outside the standard area , layout analysis is performed on the image data of the form image, and character strings are obtained from the definition data as recognition results within the standard area.” Paragraph [0013]: “When an object is detected, an annotation area in which the annotation is written is identified, and the annotation is extracted from the identified annotation area .” Paragraph [0052]: “If characters are detected in step S03 (YES in step S03), an annotation area in which characters are written is identified in step S04 because an annotation may be written outside the vicinity area 7. The annotation may be extracted by extracting character information by performing OCR processing on the annotation area, or by extracting an image of the annotation area .” Explanation: “New text area” corresponds to the annotation outside the standard area. “Determining…as target text area” corresponds to identifying annotation areas for processing. However, Miura fails to teach the limitations: “when there is a human body area of a user in the first image frame, comparing a quantity of text areas in the first image frame with a quantity of text areas in the second image frame, to detect whether a new text area exists in the first image frame,” and “when there is no new text area in the first image frame, determining a text area associated with the human body area as the target text area.” While Miura teaches detecting whether a new text area exists, comparing quantities of text areas through detecting new/unspecified annotations, and an associated text area through linking annotations to relevant standard-area items, he fails to teach a human body area and an explicit frame-to-frame comparison. However, Parmar and Bunk teach both human presence and frame comparison. Parmar and Bunk teach a human body area, stating that they “detect in the video data a presenter and tracking at least one of a movement, gesture, hand position, arm position, direction of writing of the presenter” (paragraph [0007]). The reference also teaches image frames, stating that “let p1, p2, p3, p4, p5 be the pixel values (each 1 scalar number) for the last 5 grayscale image frames at the same spatial location and m1, m2, m3, m4, m5 be the corresponding person or foreground detector mask probabilities” (paragraph [0283]). Lastly, Parmar and Bunk teach detecting new text areas, stating that “writing change detection can be done using characters detected by text detection (and/or OCR) … the 3 classes of alterations that can be defined and timestamped are addition of writing (new writing against previously blank surface), alteration of writing (for example, a correction of a mistake), and removal of writing (erase; restores surface to blankness)” (paragraphs [0291] and [0292]). This teaches human body area (presenter detection), frame-to-frame comparison, and detecting new text areas (writing addition). Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify Miura to further include detecting a human body area of a user, performing frame-to-frame comparison of text areas to detect newly added text, and associating text areas with the detected human body area, as taught by Parmar and Bunk, in order to improve contextual and relevance of text areas. A person of ordinary skill in the art would have been motivated to incorporate these teachings into Miura because doing so would allow the system to detect newly created text areas over time and associate those text areas with the user responsible for generating or interacting with them, thereby improving both accuracy and contextual relevance of text-area selection. This modification applies a known technique of human detection and temporal change detection to a similar image-processing system, yielding predictable results. Miura and Parmar and Bunk fail to teach the limitation: “when there is no human body area of a user in the first image frame, determining a text area with a largest semantic size as the target text area, wherein a semantic size of a text area is a ratio of a size of the text area to a semantic distance of the text area, and the semantic distance of the text area is a distance between the text area and a central point of the first image frame.” While Parmar and Bunk teach a human body area and Miura teaches selection based on size/geometry of text regions (paragraphs [0059], [0060], [0061]), prioritizing larger/structured regions are during detection (paragraphs [0059], [0060], [0061]), and spatial relationships (distance, size, position) ((paragraphs [0012], [0049], [0060]), there is not an explicit ratio/semantic distance definition that is disclosed. However, Su and Zhang explicitly teach ratio-based size and distance from center. Su and Zhang disclose a ratio (semantic size), stating that they “obtain a first ratio of the first diameter to the shortest side length of the first image…if the first ratio is between a first threshold and a second threshold, determine the second circular region based on a preset ratio of the second diameter to the first diameter” (paragraph [0010]). They also disclose a distance/center relationship (semantic distance), stating that they “determine the second circular region in the first image based on a circle center of the first circular region, a center of the first image, the first diameter, and the preset ratio” (paragraph [0012]), and “obtain a first distance between the center of gravity of the first image and a center of the first image” (paragraph [0019]). Lastly, they teach object-based semantic weighting, stating that “for example, a salient object in an image may be a person” (paragraph [0060]). These disclosures teach ratio-based size metrics, distance relative to center, and object-aware spatial prioritization. Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify Miura and Parmar and Bunk to determine a semantic size of a text area based on a ratio and a semantic distance based on a distance from a central point, as taught by Su and Zhang, in order to improve spatial evaluation and prioritization of text areas. A person of ordinary skill in the art would have been motivated to incorporate these teachings into Miura and Parmar and Bunk because ratio-based size evaluation and distance-from-center spatial metrics are well-known techniques for determining relative importance and positioning of visual elements, and applying such techniques would improve selection of a target text area. This modification constitutes the application of known spatial and mathematical evaluation techniques to improve a similar image processing system, yielding predictable results. Regarding Claim 19, Miura in view of Parmar and Bunk teaches the apparatus according to claim 17, and additional limitations are met as in the consideration of claim 6 above . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Wolk et. al (US9779295B2) teaches techniques for creating and manipulating software notes representative of physical notes. It would render a 102 rejection for claims 1-5. Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM ADU-JAMFI whose telephone number is (571) 272-9298. The examiner can normally be reached M-T 8:00-6:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /WILLIAM ADU-JAMFI/Examiner, Art Unit 2677 /ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677 Application/Control Number: 18/723,870 Page 2 Art Unit: 2677 Application/Control Number: 18/723,870 Page 3 Art Unit: 2677 Application/Control Number: 18/723,870 Page 4 Art Unit: 2677 Application/Control Number: 18/723,870 Page 5 Art Unit: 2677 Application/Control Number: 18/723,870 Page 6 Art Unit: 2677 Application/Control Number: 18/723,870 Page 7 Art Unit: 2677 Application/Control Number: 18/723,870 Page 8 Art Unit: 2677 Application/Control Number: 18/723,870 Page 9 Art Unit: 2677 Application/Control Number: 18/723,870 Page 10 Art Unit: 2677 Application/Control Number: 18/723,870 Page 11 Art Unit: 2677 Application/Control Number: 18/723,870 Page 12 Art Unit: 2677 Application/Control Number: 18/723,870 Page 13 Art Unit: 2677 Application/Control Number: 18/723,870 Page 14 Art Unit: 2677 Application/Control Number: 18/723,870 Page 15 Art Unit: 2677 Application/Control Number: 18/723,870 Page 16 Art Unit: 2677 Application/Control Number: 18/723,870 Page 17 Art Unit: 2677 Application/Control Number: 18/723,870 Page 18 Art Unit: 2677 Application/Control Number: 18/723,870 Page 19 Art Unit: 2677 Application/Control Number: 18/723,870 Page 20 Art Unit: 2677 Application/Control Number: 18/723,870 Page 21 Art Unit: 2677 Application/Control Number: 18/723,870 Page 22 Art Unit: 2677 Application/Control Number: 18/723,870 Page 23 Art Unit: 2677 Application/Control Number: 18/723,870 Page 24 Art Unit: 2677 Application/Control Number: 18/723,870 Page 25 Art Unit: 2677 Application/Control Number: 18/723,870 Page 26 Art Unit: 2677