DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d).
Response to Amendments
The amendments to the claims filed on 1/30/2026 have been acknowledged accepted and entered. Previously claims 1 – 17 were pending. Claims 1, 9 and 17 have been amended, and currently claims 1 – 17 are now still currently pending. Furthermore the claim amendments made to claim 17 are sufficient to overcome the 101 subject matter eligibility rejection, and thus the 101 claim rejection of claim 17 is herein withdrawn.
Response to Arguments
Applicant’s arguments with respect to claims 1 and 9 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an input/output interface module, wherein the input/output interface module is configured to display the recommended segmentation regions by overlaying them on the annotation target image” in claim 7. As per the specification (Page 15 Paragraph 1) the structure associated with the “input/output interface module” is that of a monitor, touch screen, a mouse, an electronic pen, a microphone, a keyboard, a speaker, an earphone, a headphone, or a touch pad.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 6, 7 – 9, 12, and 14 – 17 are rejected under 35 U.S.C. 103 as being unpatentable by Liu; Kangwei et al. (CN 107886512 A; translated via Espacenet; hereinafter simply referred to as Liu) in view of Pham; Quang Tuan et al. (US 20170091561 A1; hereinafter simply referred to as Pham)
Regarding independent claim 1, Liu teaches:
An annotation device comprising: at least one processor, and memory configured to store a program, wherein the at least one processor is configured to execute the instructions in the program (See ¶ 137, 166, 167, 168, and Figure 2, wherein a device is used to perform the annotation/labeling process, wherein the device comprises a processor and a memory that stores program instructions executed by the processor to execute the annotation/labeling process)
generate a plurality of superpixels in an annotation target image based on a predetermined non-parametric segmentation method (See ¶ 63, 64, 65, wherein a plurality of superpixels are generated (superpixel blocks) in an annotation target image (image to be processed) based on a predetermined non-parametric segmentation method (superpixel segmentation algorithm))
recommend segmentation regions based on outlines of the plurality of superpixels, respectively (See ¶ 102, 103 wherein the segmentation regions (labeling objects) are based on the plurality of superpixel blocks)
and perform labeling for each of the recommended segmentation regions based on a user input for labeling. (See ¶ 103, 105, 106, 108, wherein the user performs labeling on each of the recommended segmentation regions (labeled objects) via clicking or touching of the screen).
Liu does not specifically disclose the predetermined non-parametric segmentation method adaptively and automatically adjusts a count of the plurality of superpixels and a shape and size of each of the plurality of superpixels according to objects included in the annotation target image, such that the shape and size of one of the plurality of superpixels are different from the shape and size of another one of the plurality of superpixels according to the objects included in the annotation target image.
However, Pham teaches of the predetermined non-parametric segmentation method adaptively and automatically adjusts a count of the plurality of superpixels and a shape and size of each of the plurality of superpixels according to objects included in the annotation target image, (See ¶ 90 and figures 8B and 8C wherein the segmentation method adjusts the number/count of superpixels in an image (super pixel amount decided by how many regions there are of homogenous intensity) as well as the shape and size of each of the superpixels based the objects in the image (head ‘861’ in figure 8C and shoulder ‘862’ in figure 8C being comprised of differing superpixels of different shapes and sizes))
such that the shape and size of one of the plurality of superpixels are different from the shape and size of another one of the plurality of superpixels according to the objects included in the annotation target image (See ¶ 90 – 93 and figures 8A-8D wherein the plurality of superpixels are different shapes and sizes based on the objects in the image, as seen in figure 8C wherein the superpixel for the shoulder object ‘862’ and head object ‘861’ are different shapes and sizes).
As taught by Pham creating superpixels based on the objects included in the image allows for a superpixel segmentation of the background that does not belong to any of the objects in the image. (See ¶ 90 – 92 wherein the background is also given its own classification and superpixels can be created for the background that is not a part of any of the objects in the image). As both the teachings of Liu and Pham deal with the technical field of image processing using superpixels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu with Pham to teach of the predetermined non-parametric segmentation method adaptively and automatically adjusts a count of the plurality of superpixels and a shape and size of each of the plurality of superpixels according to objects included in the annotation target image, such that the shape and size of one of the plurality of superpixels are different from the shape and size of another one of the plurality of superpixels according to the objects included in the annotation target image in order to allow for superpixels to be created for the background that are not a part of any of the objects in the image.
Regarding dependent claim 4, Liu in view of Pham teaches:
Based on a user input for region modification, the at least one processor is further configured to modify at least any one segmentation region of the recommended segmentation regions. (See Liu ¶ 116 – 120, 18 – 20, 22, 49, 50 wherein the displayed recommendation segmentation regions, super pixel blocks making up the annotation frame, are updated/adjusted by user input for a new/adjusted segmentation).
Regarding dependent claim 6, Liu in view of Pham teaches:
The annotation target image is a microstructure image generated by photographing of a material of a component in a plant. (See Liu ¶ 57, 61 wherein the type of input object in the image/target image is not limited to any specific field or need and therefore images depicting a microstructure image generated by photographing material of a component in a plant are included).
Regarding dependent claim 7, Liu in view of Pham teaches:
An input/output interface module, wherein the input/output interface module is configured to display the recommended segmentation regions by overlaying them on the annotation target image. (See Liu ¶ 104, 105, 106, 107, 108, 109; wherein an input/output interface, being the hexagons that the user can interact with via clicking or touching, displays the recommended segmentation regions on the image, wherein the recommended segmentation regions are the hexagons that are interacted with).
Regarding dependent claim 8, Liu in view of Pham teaches:
The displayed recommendation segmentation regions are updated by a user input for new segmentation. (See Liu ¶ 116 – 120, 18 – 20, 22 wherein the displayed recommendation segmentation regions, super pixel blocks making up the annotation frame, are updated/adjusted by user input for a new/adjusted segmentation).
Regarding independent claim 9, claim 9 is a method claim corresponding to claim 1. Please see the discussion of claim 1 above.
Regarding dependent claim 12, claim 12 is a method claim corresponding to claim 4. Please see the discussion of claim 4 above.
Regarding dependent claim 14, claim 14 is a method claim corresponding to claim 6. Please see the discussion of claim 6 above.
Regarding dependent claim 15, claim 15 is a method claim corresponding to claim 7. Please see the discussion of claim 7 above.
Regarding dependent claim 16, claim 16 is a method claim corresponding to claim 8. Please see the discussion of claim 8 above.
Regarding dependent claim 17, Liu in view of Pham teaches:
A recording medium that is readable by a computer in which a program for performing the method according to claim 9 is recorded. (See Liu ¶ 166 – 168, 170, and 171 wherein a computer readable medium records a program used to perform the annotation method of claim 9. Please also see the discussion of claim 9 above.)
Claims 2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Liu; Kangwei et al. (CN 107886512 A; translated via Espacenet; hereinafter simply referred to as Liu) in view of Pham; Quang Tuan et al. (US 20170091561 A1; hereinafter simply referred to as Pham) and further in view of Hao; Zhi-Hui (CN 104517116 A translated via Espacenet; hereinafter simply referred to as Hao)
Regarding dependent claim 2, Liu in view of Pham teaches of:
Selecting a superpixel based on user input (See Liu ¶ 103, 105, 106, 108, 116 – 120, 18 – 20, 22, 49, and 50 wherein the user uses a user interface for super pixel selection).
Liu in view of Pham does not explicitly disclose of an additional division target superpixel; generating a plurality of clusters by clustering pixels within the additional division target superpixel based on a predetermined hierarchical and non-parametric coordinate clustering algorithm; and generating new superpixels by dividing the selected additional division target superpixel into at least two superpixels, thereby updating the plurality of superpixels to include the new superpixels.
However, Hao teaches of an additional division target superpixel; generating a plurality of clusters by clustering pixels within the additional division target superpixel based on a predetermined hierarchical and non-parametric coordinate clustering algorithm; and generating new superpixels by dividing the selected additional division target superpixel into at least two superpixels, thereby updating the plurality of superpixels to include the new superpixels. (See ¶ 70, 71, 99, 3, 68 wherein an additional division target superpixel (parent superpixel) is divided into multiple clusters using a hierarchical non-parametric coordinate clustering algorithm (k-means algorithm of a hierarchical image) which generates new superpixels thereby updating the plurality of superpixels to include the new superpixels).
As taught by Hao the use of creation of new superpixels via the division of a superpixel into clusters allows for a process to occur that determines the target areas in an image that is an undetermined image. (See ¶ 14 wherein the creation of new superpixels via the division of a superpixel into clusters is a step in the process to implement an apparatus for detection of target regions in an image to determined). As both the teachings of Liu in view of Pham and Hao deal with the technical field of image processing regarding superpixels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu in view of Pham with Hao to teach of an additional division target superpixel; generating a plurality of clusters by clustering pixels within the additional division target superpixel based on a predetermined hierarchical and non-parametric coordinate clustering algorithm; and generating new superpixels by dividing the selected additional division target superpixel into at least two superpixels, thereby updating the plurality of superpixels to include the new superpixels in order to be able to implement an apparatus for detection of target regions in an image to determined.
Regarding dependent claim 10, claim 10 is a method claim corresponding to claim 2. Please see the discussion of claim 2 above.
Claims 3 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Liu; Kangwei et al. (CN 107886512 A; translated via Espacenet; hereinafter simply referred to as Liu) in view of Pham; Quang Tuan et al. (US 20170091561 A1; hereinafter simply referred to as Pham) and further in view of Xie; Jun (US 20170372455 A1; hereinafter simply referred to as Xie)
Regarding dependent claim 3, Liu in view of Pham does not explicitly disclose:
Based on a user input for new segmentation, the at least one processor is further configured to generate a new segmentation region by merging at least two segmentation regions of the recommended segmentation regions, thereby updating the segmentation regions.
However, Xie teaches of based on a user input for new segmentation, the at least one processor is further configured to generate a new segmentation region by merging at least two segmentation regions of the recommended segmentation regions, thereby updating the segmentation regions. (See ¶ 13 and 9 wherein image segments (recommended segmentation regions) are merged together responsive to user input, thereby updating the segmentation regions.)
As taught by Xie the merging of segmentation regions responsive to user input allows for the user to have fine tuning capabilities with regards to image processing, specifically with regards to vectorization operations. (See ¶ 36 wherein the merging of segmentation regions responsive to user input allows for the user to have control over vectorization operations). As both the teachings of Liu in view of Pham and Xie deal with the technical field of image processing regarding segmentation of images, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu in view of Pham with Xie to teach of based on a user input for new segmentation, the at least one processor is further configured to generate a new segmentation region by merging at least two segmentation regions of the recommended segmentation regions, thereby updating the segmentation regions in order for the user to have fine tuning capabilities with regards to image processing, specifically with regards to vectorization operations.
Regarding dependent claim 11, claim 11 is a method claim corresponding to claim 3. Please see the discussion of claim 3 above.
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Liu; Kangwei et al. (CN 107886512 A; translated via Espacenet; hereinafter simply referred to as Liu) in view of Pham; Quang Tuan et al. (US 20170091561 A1; hereinafter simply referred to as Pham) and further in view of Shah; Ajay et al. (WO 2012020211 A1; translated via Espacenet; hereinafter simply referred to as Shah).
Regarding dependent claim 5, Liu in view of Pham does not explicitly disclose:
The at least one segmentation region is modified in units of a pixel based on the user input for region modification.
However, Shah teaches of the at least one segmentation region is modified in units of a pixel based on the user input for region modification. (See Page 3 Final Paragraph, wherein the area that is segmented is extended/modified in units of pixels, as each individual pixel is compared to the user inputted adjusted threshold, therefore the segmented area increases/is modified in units of pixels).
As taught by Shah modifying a segmentation region based on units of pixels allows for the segmentation regions to be more precise and accurately a feature of an image. (See Page 3 Final Paragraph wherein the segmentation region is extended to the boundaries of the feature of the image therefore more accurately representing the feature and its boundaries). As both the teachings of Liu in view of Pham and Shah deal with the technical field of image processing regarding the segmentation of pixels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Liu in view of Pham with Shah to teach of the at least one segmentation region is modified in units of a pixel based on the user input for region modification in order for the segmentation regions to more accurately represent the features of an image by extending to the boundaries of the feature of the image.
Regarding dependent claim 13, claim 13 is a method claim corresponding to claim 5. Please see the discussion of claim 5 above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEJANDRO HERNANDEZ whose telephone number is (703)756-1876. The examiner can normally be reached M-F 8 am - 5 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEJANDRO HERNANDEZExaminer, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights