DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d).
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/29/2026 has been entered.
Response to Amendments
The amendments to the claims filed on 07/29/2026 have been acknowledged accepted and entered. Previously claims 1 – 17 were pending. Claims 1 and 9 have been amended, claims 4, 5, 12, and 13 have been canceled, and now claims 1 – 3, 6 – 11, and 14 – 17 are still currently pending.
Response to Arguments
Applicant’s arguments with respect to claims 1 and 9 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Specifically the arguments directed towards the details required for the selection information and shape modification information limitations was persuasive, and thus a new ground of rejections has been applied.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an input/output interface module, wherein the input/output interface module is configured to display the recommended segmentation regions by overlaying them on the annotation target image” in claim 7. As per the specification (Page 15 Paragraph 1) the structure associated with the “input/output interface module” is that of a monitor, touch screen, a mouse, an electronic pen, a microphone, a keyboard, a speaker, an earphone, a headphone, or a touch pad.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6 – 9, and 14 – 17 are rejected under 35 U.S.C. 103 as being unpatentable by Liu; Kangwei et al. (CN 107886512 A; translated via Espacenet; hereinafter simply referred to as Liu) in view of Pham; Quang Tuan et al. (US 20170091561 A1; hereinafter simply referred to as Pham) further in view of Nishida; Koji et al. (WO 2021235245 A1; hereinafter simply referred to as Nishida; translated via Espacenet).
Regarding independent claim 1, Liu teaches:
An annotation device comprising: at least one processor, and memory configured to store a program, wherein the at least one processor is configured to execute the instructions in the program (See ¶ 137, 166, 167, 168, and Figure 2, wherein a device is used to perform the annotation/labeling process, wherein the device comprises a processor and a memory that stores program instructions executed by the processor to execute the annotation/labeling process)
generate a plurality of superpixels in an annotation target image based on a predetermined non-parametric segmentation method (See ¶ 63, 64, 65, wherein a plurality of superpixels are generated (superpixel blocks) in an annotation target image (image to be processed) based on a predetermined non-parametric segmentation method (superpixel segmentation algorithm))
recommend a plurality of segmentation regions based on outlines of the plurality of superpixels, respectively wherein each of the plurality of segmentation regions corresponds to a group of pixels forming a segmentation region shape (See ¶ 102, 103 wherein the segmentation regions (labeling objects) are based on the plurality of superpixel blocks, wherein the superpixel blocks are necessarily made up of pixels (groups of pixels) that form some kind of shape (segmentation region shape))
and perform labeling for each of the recommended plurality of segmentation regions based on a user input for labeling. (See ¶ 103, 105, 106, 108, wherein the user performs labeling on each of the plurality of recommended segmentation regions (labeled objects) via clicking or touching of the screen).
Liu does not specifically disclose the predetermined non-parametric segmentation method adaptively and automatically adjusts a count of the plurality of superpixels and a shape and size of each of the plurality of superpixels according to objects included in the annotation target image, such that the shape and size of one of the plurality of superpixels are different from the shape and size of another one of the plurality of superpixels according to the objects included in the annotation target image.
However, Pham teaches of the predetermined non-parametric segmentation method adaptively and automatically adjusts a count of the plurality of superpixels and a shape and size of each of the plurality of superpixels according to objects included in the annotation target image, (See ¶ 90 and figures 8B and 8C wherein the segmentation method adjusts the number/count of superpixels in an image (super pixel amount decided by how many regions there are of homogenous intensity) as well as the shape and size of each of the superpixels based the objects in the image (head ‘861’ in figure 8C and shoulder ‘862’ in figure 8C being comprised of differing superpixels of different shapes and sizes))
such that the shape and size of one of the plurality of superpixels are different from the shape and size of another one of the plurality of superpixels according to the objects included in the annotation target image (See ¶ 90 – 93 and figures 8A-8D wherein the plurality of superpixels are different shapes and sizes based on the objects in the image, as seen in figure 8C wherein the superpixel for the shoulder object ‘862’ and head object ‘861’ are different shapes and sizes).
As taught by Pham creating superpixels based on the objects included in the image allows for a superpixel segmentation of the background that does not belong to any of the objects in the image. (See ¶ 90 – 92 wherein the background is also given its own classification and superpixels can be created for the background that is not a part of any of the objects in the image). As both the teachings of Liu and Pham deal with the technical field of image processing using superpixels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu with Pham to teach of the predetermined non-parametric segmentation method adaptively and automatically adjusts a count of the plurality of superpixels and a shape and size of each of the plurality of superpixels according to objects included in the annotation target image, such that the shape and size of one of the plurality of superpixels are different from the shape and size of another one of the plurality of superpixels according to the objects included in the annotation target image in order to allow for superpixels to be created for the background that are not a part of any of the objects in the image.
Liu in view of Pham does not explicitly disclose receive a user input for region modification, which includes selection information selecting one segmentation region from among the recommended plurality of segmentation regions and shape modification information modifying the group of pixels corresponding to the selected one segmentation region, and modify the segmentation region shape of the selected one segmentation region from among the plurality of segmentation regions according to the modified group of pixels corresponding to the selection segmentation region.
However, Nishida teaches of receive a user input for region modification, which includes selection information selecting one segmentation region from among the recommended plurality of segmentation regions and shape modification information modifying the group of pixels corresponding to the selected one segmentation region, (See ¶ 234, 244, 245 wherein a user uses the object adjustment unit to change the shape of an object (selected segmentation region) via the addition/deletion of superpixels (changing the shape of an object via the changing of the superpixels made up of groups of pixels))
and modify the segmentation region shape of the selected one segmentation region from among the plurality of segmentation regions according to the modified group of pixels corresponding to the selection segmentation region. (See ¶ 234 – 238 wherein the segmentation region shape (shape of object) is modified based on the user selecting the object (selected segmentation region) and adding/deleting superpixels to the object, wherein the adding/deletion of superpixels changes the shape, and wherein the superpixels are made up of pixels and therefore the group of pixels are modified)
As taught by Nishida the modification of the segmentation shape data via user input allows for updated labels to be output by the output unit corresponding to the information input from the user. (See ¶ 237 wherein the labels for the objects (which have been adjusted by the user) are updated based on the input operations received from the user). As both the teachings of Liu in view of Pham and Nishida deal with the technical field of image processing regarding the segmentation of images, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu in view of Pham with Nishida to teach of receive a user input for region modification, which includes selection information selecting one segmentation region from among the recommended plurality of segmentation regions and shape modification information modifying the group of pixels corresponding to the selected one segmentation region, and modify the segmentation region shape of the selected one segmentation region from among the plurality of segmentation regions according to the modified group of pixels corresponding to the selection segmentation region in order for updated labels for objects to be generated.
Regarding dependent claim 6, Liu in view of Pham and Nishida teaches:
The annotation target image is a microstructure image generated by photographing of a material of a component in a plant. (See Liu ¶ 57, 61 wherein the type of input object in the image/target image is not limited to any specific field or need and therefore images depicting a microstructure image generated by photographing material of a component in a plant are included).
Regarding dependent claim 7, Liu in view of Pham and Nishida teaches:
An input/output interface module, wherein the input/output interface module is configured to display the recommended segmentation regions by overlaying them on the annotation target image. (See Liu ¶ 104, 105, 106, 107, 108, 109; wherein an input/output interface, being the hexagons that the user can interact with via clicking or touching, displays the recommended segmentation regions on the image, wherein the recommended segmentation regions are the hexagons that are interacted with).
Regarding dependent claim 8, Liu in view of Pham and Nishida teaches:
The displayed recommendation segmentation regions are updated by a user input for new segmentation. (See Liu ¶ 116 – 120, 18 – 20, 22 wherein the displayed recommendation segmentation regions, super pixel blocks making up the annotation frame, are updated/adjusted by user input for a new/adjusted segmentation).
Regarding independent claim 9, claim 9 is a method claim corresponding to claim 1. Please see the discussion of claim 1 above.
Regarding dependent claim 14, claim 14 is a method claim corresponding to claim 6. Please see the discussion of claim 6 above.
Regarding dependent claim 15, claim 15 is a method claim corresponding to claim 7. Please see the discussion of claim 7 above.
Regarding dependent claim 16, claim 16 is a method claim corresponding to claim 8. Please see the discussion of claim 8 above.
Regarding dependent claim 17, Liu in view of Pham and Nishida teaches:
A recording medium that is readable by a computer in which a program for performing the method according to claim 9 is recorded. (See Liu ¶ 166 – 168, 170, and 171 wherein a computer readable medium records a program used to perform the annotation method of claim 9. Please also see the discussion of claim 9 above.)
Claims 2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Liu; Kangwei et al. (CN 107886512 A; translated via Espacenet; hereinafter simply referred to as Liu) in view of Pham; Quang Tuan et al. (US 20170091561 A1; hereinafter simply referred to as Pham) further in view of Nishida; Koji et al. (WO 2021235245 A1; hereinafter simply referred to as Nishida; translated via Espacenet) and further in view of Hao; Zhi-Hui (CN 104517116 A translated via Espacenet; hereinafter simply referred to as Hao)
Regarding dependent claim 2, Liu in view of Pham and Nishida teaches of:
Selecting a superpixel based on user input (See Liu ¶ 103, 105, 106, 108, 116 – 120, 18 – 20, 22, 49, and 50 wherein the user uses a user interface for super pixel selection).
Liu in view of Pham does not explicitly disclose of an additional division target superpixel; generating a plurality of clusters by clustering pixels within the additional division target superpixel based on a predetermined hierarchical and non-parametric coordinate clustering algorithm; and generating new superpixels by dividing the selected additional division target superpixel into at least two superpixels, thereby updating the plurality of superpixels to include the new superpixels.
However, Hao teaches of an additional division target superpixel; generating a plurality of clusters by clustering pixels within the additional division target superpixel based on a predetermined hierarchical and non-parametric coordinate clustering algorithm; and generating new superpixels by dividing the selected additional division target superpixel into at least two superpixels, thereby updating the plurality of superpixels to include the new superpixels. (See ¶ 70, 71, 99, 3, 68 wherein an additional division target superpixel (parent superpixel) is divided into multiple clusters using a hierarchical non-parametric coordinate clustering algorithm (k-means algorithm of a hierarchical image) which generates new superpixels thereby updating the plurality of superpixels to include the new superpixels).
As taught by Hao the use of creation of new superpixels via the division of a superpixel into clusters allows for a process to occur that determines the target areas in an image that is an undetermined image. (See ¶ 14 wherein the creation of new superpixels via the division of a superpixel into clusters is a step in the process to implement an apparatus for detection of target regions in an image to determined). As both the teachings of Liu in view of Pham and Nishida and Hao deal with the technical field of image processing regarding superpixels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu in view of Pham and Nishida with Hao to teach of an additional division target superpixel; generating a plurality of clusters by clustering pixels within the additional division target superpixel based on a predetermined hierarchical and non-parametric coordinate clustering algorithm; and generating new superpixels by dividing the selected additional division target superpixel into at least two superpixels, thereby updating the plurality of superpixels to include the new superpixels in order to be able to implement an apparatus for detection of target regions in an image to determined.
Regarding dependent claim 10, claim 10 is a method claim corresponding to claim 2. Please see the discussion of claim 2 above.
Claims 3 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Liu; Kangwei et al. (CN 107886512 A; translated via Espacenet; hereinafter simply referred to as Liu) in view of Pham; Quang Tuan et al. (US 20170091561 A1; hereinafter simply referred to as Pham) further in view of Nishida; Koji et al. (WO 2021235245 A1; hereinafter simply referred to as Nishida; translated via Espacenet) and further in view of Xie; Jun (US 20170372455 A1; hereinafter simply referred to as Xie)
Regarding dependent claim 3, Liu in view of Pham and Nishida does not explicitly disclose:
Based on a user input for new segmentation, the at least one processor is further configured to generate a new segmentation region by merging at least two segmentation regions of the recommended segmentation regions, thereby updating the segmentation regions.
However, Xie teaches of based on a user input for new segmentation, the at least one processor is further configured to generate a new segmentation region by merging at least two segmentation regions of the recommended segmentation regions, thereby updating the segmentation regions. (See ¶ 13 and 9 wherein image segments (recommended segmentation regions) are merged together responsive to user input, thereby updating the segmentation regions.)
As taught by Xie the merging of segmentation regions responsive to user input allows for the user to have fine tuning capabilities with regards to image processing, specifically with regards to vectorization operations. (See ¶ 36 wherein the merging of segmentation regions responsive to user input allows for the user to have control over vectorization operations). As both the teachings of Liu in view of Pham and Nishida and Xie deal with the technical field of image processing regarding segmentation of images, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu in view of Pham and Nishida with Xie to teach of based on a user input for new segmentation, the at least one processor is further configured to generate a new segmentation region by merging at least two segmentation regions of the recommended segmentation regions, thereby updating the segmentation regions in order for the user to have fine tuning capabilities with regards to image processing, specifically with regards to vectorization operations.
Regarding dependent claim 11, claim 11 is a method claim corresponding to claim 3. Please see the discussion of claim 3 above.
Prior Art Made of Record
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure and is as follows:
U.S. Patent Application No. US 20220019730 A1 (Sharma) discloses the use of a user interface superpixel tool allowing a user to change the shape of segmentations within an image by changing the segment sizes and using a pen tool and eraser tool to adjust boundaries of the segmentation. (Sharma [0065])
Furthermore please See attached PTO-892.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEJANDRO HERNANDEZ whose telephone number is (703)756-1876. The examiner can normally be reached M-F 8 am - 5 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEJANDRO HERNANDEZExaminer, Art Unit 2661
/AARON W CARTER/Primary Examiner, Art Unit 2661
Prosecution Insights