DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “generic face image quality assessment network…to receive”, “fine-tuned vision transformer configured to receive”, “degradation extraction network configured to identify”, “landmark detection network configured to identify”, “transformer decoder configured to process”, “channel attention block…configured to emphasize” and “Swin Transformer…configured to refine” in claims 1, 9 and 10.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (“DSL-FIQA: Assessing Facial Image Quality via Dual-Set Degradation Learning and Landmark-Guided Transformer”).
Regarding claim 1, Chen et al. discloses an image quality assessment network, comprising:
a generic face image quality assessment network (“The model contains a core GFIQA network” at Figure 2 description, line 1) having an input configured to receive an image and crop the image into a plurality of patches (“In our approach, face images are cropped into several patches” at Figure 2 description, line 2);
a vision transformer configured to receive and process the plurality of patches (“The image initially undergoes feature extraction [73] via a pre-trained Vision Transformer (ViT)” at section 3.1, line 4);
a degradation extraction network configured to identify and isolate perceptual degradations of the image and provide a degradation representation of image quality degradations of the image (“In parallel, a dedicated module identifies and isolates perceptual degradations within the image, providing a nuanced representation of image quality degradations” at section 3.1, line 10);
a landmark detection network configured to identify facial key points of the image (“A landmark detection network identifies facial key points, influencing the regional confidence evaluation and ensuring that essential facial features improve the final quality score” at section 3.1, third to last paragraph); and
a transformer decoder configured to process the fine-tuned ViT processed patches, the degradation representation, and the facial key points (“The degradation features, once extracted, are integrated with the outputs from the Swin Transformer within a transformer decoder” at section 3.1, line 14; “The encoded information is subsequently concatenated with the features processed by the Transformer Decoder, feeding into the regional confidence branch” at section 3.3, paragraph 3, line 1), and generate a score indicative of a quality of the image (“Finally, these outputs are combined through a weighted sum to determine the overall quality score of the image” at section 3.1, line 22).
Chen et al. does not explicitly disclose that the vision transformed is fine-tuned.
However, it is well known in the art to take a pre-trained transformer, such as a ViT, and refine the weights prior to full implementation for a new system. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to fine-tune the ViT to optimize it for use with facial images.
Regarding claim 12, Chen et al. discloses a method of using a generic face image quality assessment network having an input configured to receive an image and crop the input image into a plurality of patches, a vision transformer (ViT) configured to receive and process the plurality of patches, a degradation extraction network configured to identify and isolate perceptual degradations of the image and provide a degradation representation of image quality degradations of the image, a landmark detection network configured to identify facial key points of the image, and a transformer decoder configured to process the processed patches, the degradation representation, and the facial key points from the landmark detection network (“Figure 2. Overview of our proposed model. The model contains a core GFIQA network, a degradation extraction network, and a landmark detection network. In our approach, face images are cropped into several patches to fit the input size requirements of the pre-trained ViT feature extractor (See Sec. 3.1). Each patch is then processed individually, and their Mean Opinion Scores (MOS) are averaged to determine the final quality score. For clarity in the figure, the segmentation of the image into patches is not shown.” at Figure 2), the method comprising the steps of:
the GFIQA network receiving an image and cropping the image into a plurality of patches (“In our approach, face images are cropped into several patches” at Figure 2 description, line 2);
the ViT receiving and processing the plurality of patches (“The image initially undergoes feature extraction [73] via a pre-trained Vision Transformer (ViT)” at section 3.1, line 4);
the degradation extraction network identifying and isolating perceptual degradations of the image and providing a degradation representation of image quality degradations of the image (“In parallel, a dedicated module identifies and isolates perceptual degradations within the image, providing a nuanced representation of image quality degradations” at section 3.1, line 10);
the landmark detection network identifying the facial key points of the image (“A landmark detection network identifies facial key points, influencing the regional confidence evaluation and ensuring that essential facial features improve the final quality score” at section 3.1, third to last paragraph); and
the transformer decoder processing the processed patches, the degradation representation, and the facial key points from the landmark detection network (“The degradation features, once extracted, are integrated with the outputs from the Swin Transformer within a transformer decoder” at section 3.1, line 14; “The encoded information is subsequently concatenated with the features processed by the Transformer Decoder, feeding into the regional confidence branch” at section 3.3, paragraph 3, line 1), and generates a score indicative of a quality of the image (“Finally, these outputs are combined through a weighted sum to determine the overall quality score of the image” at section 3.1, line 22).
Chen et al. does not explicitly disclose that the vision transformed is fine-tuned.
However, it is well known in the art to take a pre-trained transformer, such as a ViT, and refine the weights prior to full implementation for a new system. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to fine-tune the ViT to optimize it for use with facial images.
Regarding claim 20, Chen et al. discloses a non-transitory computer readable storage medium that stores instructions that when executed by a processor cause the processor to process an image (implied that the system is carried out using a programmed computer) using a method by performing the steps of:
a generic face image quality assessment network (“The model contains a core GFIQA network” at Figure 2 description, line 1) receiving an image and cropping the image into a plurality of patches (“In our approach, face images are cropped into several patches” at Figure 2 description, line 2);
a vision transformer receiving and processing the plurality of patches (“The image initially undergoes feature extraction [73] via a pre-trained Vision Transformer (ViT)” at section 3.1, line 4);
a degradation extraction network identifying and isolating perceptual degradations of the image and provide a degradation representation of image quality degradations of the image (“In parallel, a dedicated module identifies and isolates perceptual degradations within the image, providing a nuanced representation of image quality degradations” at section 3.1, line 10);
a landmark detection network identifying facial key points of the image (“A landmark detection network identifies facial key points, influencing the regional confidence evaluation and ensuring that essential facial features improve the final quality score” at section 3.1, third to last paragraph); and
a transformer decoder processing the fine-tuned ViT processed patches, the degradation representation, and the facial key points (“The degradation features, once extracted, are integrated with the outputs from the Swin Transformer within a transformer decoder” at section 3.1, line 14; “The encoded information is subsequently concatenated with the features processed by the Transformer Decoder, feeding into the regional confidence branch” at section 3.3, paragraph 3, line 1), and generate a score indicative of a quality of the image (“Finally, these outputs are combined through a weighted sum to determine the overall quality score of the image” at section 3.1, line 22).
Chen et al. does not explicitly disclose that the vision transformed is fine-tuned.
However, it is well known in the art to take a pre-trained transformer, such as a ViT, and refine the weights prior to full implementation for a new system. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to fine-tune the ViT to optimize it for use with facial images.
Regarding claims 2 and 13, Chen et al. discloses a network and method wherein each of the patches are configured to be processed independently (“Each patch is then processed individually” at Figure 2 description, line 3).
Regarding claims 3, 14 and 15, Chen et al. discloses a network and method wherein the degradation extraction network includes an encoder configured to encode the patches into degradation representations for a query (“In this framework, patches extracted from the same image are positive samples, while those from different images are negative samples. The patches are encoded into degradation representations (x, x+, and x−) for the query, positive, and negative samples” at section 3.2.1, line 3).
Regarding claims 4 and 16, Chen et al. discloses a network and method wherein the landmark detection network is configured to influence regional confidence evaluation of essential facial features of the image to improve the score (“A landmark detection network identifies facial key points, influencing the regional confidence evaluation and ensuring that essential facial features improve the final quality score” at section 3.1, third to last paragraph).
Regarding claims 5 and 17, Chen et al. discloses a network and method wherein the score is an average of scores of independently processed patches (“Instead, we crop the image, process each part independently, and average the resulting MOS predictions for a consolidated image quality score” at section 3.1, second to last paragraph, line 5).
Regarding claim 6, Chen et al. discloses a network wherein each processed patch comprises a mean opinion score (“their Mean Opinion Scores (MOS) are averaged” at Figure 2 description, line 3).
Regarding claims 7 and 18, Chen et al. discloses a network and method wherein the GFIQA network includes an extractor configured to crop the input images to fit fixed input dimensions of the fine-tuned ViT (“In our approach, face images are cropped into several patches to fit the input size requirements of the pre-trained ViT feature extractor” at Figure 2 description, line 2).
Regarding claims 8 and 19, Chen et al. discloses a network and method wherein the degradation extraction network is configured to operate in parallel and simultaneously identify and isolate the perceptual degradations of the image while the GFIQA network processes the image (“Feature Extraction and Refinement: The image initially undergoes feature extraction [73] via a pre-trained Vision Transformer (ViT) [14], followed by a Channel Attention Block [23] that emphasizes relevant inter-channel dependencies. Subsequently, a Swin Transformer Block [40] refines these features, capturing subtle image details. Degradation Extraction: In parallel, a dedicated module identifies and isolates perceptual degradations within the image, providing a nuanced representation of image quality degradations. Feature Integration” at section 3.1, line 4).
Regarding claim 9, Chen et al. discloses a network further comprising a channel attention block coupled to the fine-tuned ViT and configured to emphasize relevant inter-channel dependencies (“The image initially undergoes feature extraction [73] via a pre-trained Vision Transformer (ViT) [14], followed by a Channel Attention Block [23] that emphasizes relevant inter-channel dependencies” at section 3.1, line 4).
Regarding claim 10, Chen et al. discloses a network further comprising a Swin Transformer coupled to the attention block and configured to refine features and capture subtle image details (“Subsequently, a Swin Transformer Block [40] refines these features, capturing subtle image details” at section 3.1, line 8).
Regarding claim 11, Chen et al. discloses a network wherein the transformer decoder comprises two multi-layer perceptron branches, including a first branch configured to predict a regional confidence, and a second branch configured to estimate a regional quality score (“The combined features are then directed into two MLP branches. The first branch predicts the regional confidence, while the second estimates the regional quality score” at section 3.1, line 19).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 9:30-5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/ Primary Examiner, Art Unit 2672
Prosecution Insights