Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to the communication filed on 07/29/2026.
Claims 1, 4-8, 11-15, and 18-20 have been currently amended. Claims 2-3, 9-10, and 16-17 have been canceled. Claims 1, 4-8, 11-15, and 18-20 are pending.
Response to Arguments
Applicant’s arguments filed on 07/29/2026 on pages 10-13, under REMARKS with respect to 35 U.S.C. 102 and 103 claim rejections to claims 1-20 have been fully considered and are persuasive. The rejections to the claims have been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of US 2010/0189354 A1.
Information Disclosure Statement
The information disclosure statement (IDS) filed on 07/10/2026 has been considered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1, 4-8, 11-15, and 18-20 are rejected under 35 § U.S.C. 103 as being obvious over US 2015/0161474 A1 to JABER et al. (hereinafter “JABER”) in view of US 2010/0189354 A1 to DE CAMPOS ET AL. (hereinafter “CAMPOS”).
As per claim 1, JABER discloses a content recognition method, performed by a computer device (a computing system adapted to perform an image processing method directed towards recognizing specific content or activity in image/video frames; abstract; fig 1; paragraphs [0040], [0047-0051]), the method comprising: acquiring an image (the computing system is adapted to receive an input image; fig 9; paragraphs [0103], [0108]); recognizing a target region from the image by performing saliency detection on the image (target regions are identified in the input image using saliency algorithms and edge detection features among others; paragraphs [0043], [0050-0052], [0103], [0108]); extracting a key point feature representation corresponding to the image key points in the target region (key points such as image edges and others are extracted using the feature extraction algorithms in order to identify edged and other key points in the images and image regions; fig 2; paragraphs [0053-0054]); performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation (the system is adapted to perform image pooling by grouping the regions of interest into categories based on identified features using the feature extraction portion and the feature/content recognition models that were trained to identify specific features relating to specific contents; fig 2; paragraphs [0053-0054], [0074]); and recognizing a category of target content comprised in the target region based on the global feature representation and the second local feature representation (the computing system is able to recognize content using anyone of models A-x and the models are adapted to sort the image content by content type based on preselected feature values and thresholds learned by the model; fig 2; paragraphs [0053-0054], [0074-0078]). JABER fails to disclose extracting an image feature representation corresponding to the target region; extracting image key points from the target region based on a distribution rule of pixel points in the target region; down sampling the image feature representation based on the target region to obtain a first local feature representation; performing feature concatenation on the key point feature representation and the first local feature representation to obtain a second local feature representation, wherein a size of the second local feature representation is a sum of that of the key point feature representation and the first local feature representation.
CAMPOS discloses extracting an image feature representation corresponding to the target region (taking input images and extracting feature containing patches by labeling them yes or no and these patches act as target regions; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]); extracting image key points from the target region based on a distribution rule of pixel points in the target region (the system extracts sets of low level features called descriptors are extracted from the image without consideration to the relevance information or saliency maps which are updated using the weighted parameters related to the features in the patched image areas; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]); down sampling the image feature representation based on the target region to obtain a first local feature representation (the computing system is adapted to sample images at a variety of sampling scales during training and would be adjusted so the images are down sampled during the described steps; figs 1-2, 4B-5; paragraphs [0034], [0041-0048], [0059], [0080], [0125-0130]); performing feature concatenation on the key point feature representation and the first local feature representation to obtain a second local feature representation, wherein a size of the second local feature representation is a sum of that of the key point feature representation and the first local feature representation (at step 114 the high level extracted features from the image patch region are weighted based on computer generated parameter weighting factors and the applied weights are normalized to sum to a predetermined value such that the values are concatenated as a vector value; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to modify JABER to have an image feature representation corresponding to the target region of CAMPOS reference. The Suggestion/motivation for doing so would have been to provide the ability to combine in a single image a set of orderless feature vectors of an input image and produce a weighted image feature map as suggested and provided by figure 4B and paragraph [0049] of CAMPOS. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine CAMPOS with JABER to obtain the invention as specified in claim 1.
As per claim 4, JABER in view of CAMPOS discloses the method according to claim 1. Modified JABER further discloses wherein the down sampling the image feature representation based on the target region to obtain a first local feature representation comprises: performing sparse sampling on the image feature representation based on the target region to obtain a sparse sampling result (preprocessing module leverages an invariant feature identification algorithm, preferably one that operates quickly on the target data, to generate a set of invariant features from the digital representation and is substantially similar to sparse sampling in application; paragraphs [0023], [0043-0047]); and performing pooling on the sparse sampling result to obtain the first local feature representation (the system is adapted to perform image pooling by grouping the regions of interest into categories based on identified features using the feature extraction portion and the feature/content recognition models that were trained to identify specific features relating to specific contents; fig 2; paragraphs [0053-0054], [0074]).
As per claim 5, JABER in view of CAMPOS discloses the method according to claim 1. JABER fails to disclose wherein the recognizing a category of target content comprised in the target region based on the global feature representation and the second local feature representation comprises: acquiring a content category library, the content category library comprising a preset set of n categories, where n is a positive integer; matching the global feature representation with n categories in the content category library respectively to obtain k candidate categories in the content category library that match the global feature representation, where 0<k<n and k is an integer; performing category sorting on the k candidate categories based on the second local feature representation to obtain a category sorting result; and obtaining a recognition category corresponding to the target content according to the category sorting result.
CAMPOS discloses wherein the recognizing a category of target content comprised in the target region based on the global feature representation and the second local feature representation comprises (a computing system and method for taking an image creating a representation of that image such as a reconstruction or feature map and recognizing an object/content in the image within a region of interest and determining a category of a predetermined number of categories for the object; abstract figs 1-2; paragraphs [0059]): acquiring a content category library, the content category library comprising a preset set of n categories, where n is a positive integer (the memory of the computing device would serve as the content category library and comprises a predetermined number of classes n and n would be positive more than zero; abstract figs 1-2; paragraphs [0059]; claim 6); matching the global feature representation with n categories in the content category library respectively to obtain k candidate categories in the content category library that match the global feature representation, where 0<k<n and k is an integer (using the recognition model the extracted features would be sorted into the n classes and there would comprise k greater than 0 but less than one because each object is sorted into its nearest class making in this example k’s value always equal to one which meets the claimed range of k values; abstract figs 1-2; paragraphs [0059]; claim 6); performing category sorting on the k candidate categories based on the second local feature representation to obtain a category sorting result (the regions of image comprising objects/content is then sorted by the model in the determined class; abstract figs 1-2; paragraphs [0059]; claim 6); and obtaining a recognition category corresponding to the target content according to the category sorting result (via the display and graphic user interface the class the image feature of the image feature map would be observed and the result would be displayed to users using the display; fig 2; paragraphs [0030-0031], [0048], [0112]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to modify JABER to have obtaining a recognition category corresponding to the target content according to the category sorting result of CAMPOS reference. The Suggestion/motivation for doing so would have been to provide the ability to identify high level image features using low level image features as suggested by paragraph [0048] of CAMPOS. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine CAMPOS with JABER to obtain the invention as specified in claim 5.
As per claim 6, JABER in view of CAMPOS discloses the method according to claim 1. Modified JABER further discloses wherein the performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation comprises: inputting the target region into a content recognition model, and outputting the image feature representation, wherein the content recognition model is configured to perform deep feature extraction on the target region (the computing system is adapted to input image content to the computing system which then applies a variety of recognition models to determine an image representation 121 of a region of the input image and further extract key points and regions of interest; fig 2; paragraphs [0053-0054], [0074-0078]); and performing GeM on the image feature representation to obtain the global feature representation (the computing system is adapted to cluster features together of an image region the clusters can first be identified, and then analyze the saliency measures of the FAST features within each cluster to aid during classification, in these situations, a low-average saliency measure is a number indicating that a corresponding feature is not likely to be very useful in the analysis of a cluster can be an indication of a 3D object while a relatively high-average saliency measure or a large number indicating the corresponding feature is likely to be useful in the analysis of a cluster can indicate a region of text; paragraph [0065]; NOTE GEM pooling is defined as according to google: a trainable pooling technique which is described in paragraph [0065]).
As per claim 7, JABER in view of CAMPOS discloses the method according to claim 1. Modified JABER further discloses wherein the performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation comprises: performing mean-pooling on the image feature representation to obtain a first global feature representation (the computing system is adapted to use averaging during utilization of a salience algorithm to determine clusters proper category; paragraph [0065]); performing max-pooling on the image feature representation to obtain a second global feature representation (the computing system defines the clustering criteria and the selection criteria 125 could include a low density threshold possibly representing a minimum density (i.e., the minimum density necessary for a particular recognition module to be effective or to be preferable over other recognition modules), high density threshold possibly representing a maximum density (i.e., the maximum density for which a particular recognition module is considered to be effective or preferable over other recognition modules), and/or a feature density range applicable for each recognition module A-N (i.e., the feature density range between a minimum and maximum in which a particular recognition module is deemed most effective and/or preferred over other available recognition modules; paragraphs [0073]); performing GeM on the image feature representation to obtain a third global feature representation (the computing system is adapted to cluster features together of an image region the clusters can first be identified, and then analyze the saliency measures of the FAST features within each cluster to aid during classification, in these situations, a low-average saliency measure (e.g., a number indicating that a corresponding feature is not likely to be very useful in the analysis) of a cluster can be an indication of a 3D object while a relatively high-average saliency measure (e.g., a large number indicating the corresponding feature is likely to be useful in the analysis) of a cluster can indicate a region of text; paragraph [0065]; NOTE GEM pooling is defined as according to google: a trainable pooling technique which is described in paragraph [0065]); and performing feature concatenation on the first global feature representation, the second global feature representation, and the third global feature representation to obtain the global feature representation (feature concatenation occurs during the various model training and setting specific thresholds for features that lead to specific outcomes or content types to be sorted in the cluster/category assigned to those feature values this occurs because the same model is adapted to recognize the same features grouping/clustering image regions having the same features together; fig 2; paragraphs [0053-0054], [0074-0078]).
As per claim 8, JABER discloses a computer device comprising a processor and a memory (a computing system comprising a processor and memory adapted to perform an image processing method directed towards recognizing specific content or activity in image/video frames; abstract; fig 1; paragraphs [0040], [0047-0051]; claim 1), the memory storing at least one program, the at least one program being loaded and executed by the processor and causing the computer device to implement a content recognition method including (the computing system comprising the processor and memory adapted to perform store and execute instructions, programs and data to perform an image processing method directed towards recognizing specific content or activity in image/video frames; abstract; fig 1; paragraphs [0040], [0047-0051]; claim 1): acquiring an image (the computing system is adapted to receive an input image; fig 9; paragraphs [0103], [0108]); recognizing a target region from the image by performing saliency detection on the image (target regions are identified in the input image using saliency algorithms and edge detection features among others; paragraphs [0043], [0050-0052], [0103], [0108]); extracting a key point feature representation corresponding to the image key points in the target region (key points such as image edges and others are extracted using the feature extraction algorithms in order to identify edged and other key points in the images and image regions; fig 2; paragraphs [0053-0054]); performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation (the system is adapted to perform image pooling by grouping the regions of interest into categories based on identified features using the feature extraction portion and the feature/content recognition models that were trained to identify specific features relating to specific contents; fig 2; paragraphs [0053-0054], [0074]); and recognizing a category of target content comprised in the target region based on the global feature representation and the second local feature representation (the computing system is able to recognize content using anyone of models A-x and the models are adapted to sort the image content by content type based on preselected feature values and thresholds learned by the model; fig 2; paragraphs [0053-0054], [0074-0078]). JABER fails to disclose extracting an image feature representation corresponding to the target region; extracting image key points from the target region based on a distribution rule of pixel points in the target region; down sampling the image feature representation based on the target region to obtain a first local feature representation; performing feature concatenation on the key point feature representation and the first local feature representation to obtain a second local feature representation, wherein a size of the second local feature representation is a sum of that of the key point feature representation and the first local feature representation.
CAMPOS discloses extracting an image feature representation corresponding to the target region (taking input images and extracting feature containing patches by labeling them yes or no and these patches act as target regions; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]); extracting image key points from the target region based on a distribution rule of pixel points in the target region (the system extracts sets of low level features called descriptors are extracted from the image without consideration to the relevance information or saliency maps which are updated using the weighted parameters related to the features in the patched image areas; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]); down sampling the image feature representation based on the target region to obtain a first local feature representation (the computing system is adapted to sample images at a variety of sampling scales during training and would be adjusted so the images are down sampled during the described steps; figs 1-2, 4B-5; paragraphs [0034], [0041-0048], [0059], [0080], [0125-0130]); performing feature concatenation on the key point feature representation and the first local feature representation to obtain a second local feature representation, wherein a size of the second local feature representation is a sum of that of the key point feature representation and the first local feature representation (at step 114 the high level extracted features from the image patch region are weighted based on computer generated parameter weighting factors and the applied weights are normalized to sum to a predetermined value such that the values are concatenated as a vector value; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to modify JABER to have an image feature representation corresponding to the target region of CAMPOS reference. The Suggestion/motivation for doing so would have been to provide the ability to combine in a single image a set of orderless feature vectors of an input image and produce a weighted image feature map as suggested and provided by figure 4B and paragraph [0049] of CAMPOS. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine CAMPOS with JABER to obtain the invention as specified in claim 8.
As per claim 11, JABER in view of CAMPOS discloses the computer device according to claim 8. Modified JABER further discloses wherein the down sampling the image feature representation based on the target region to obtain a first local feature representation comprises: performing sparse sampling on the image feature representation based on the target region to obtain a sparse sampling result (preprocessing module leverages an invariant feature identification algorithm, preferably one that operates quickly on the target data, to generate a set of invariant features from the digital representation and is substantially similar to sparse sampling in application; paragraphs [0023], [0043-0047]); and performing pooling on the sparse sampling result to obtain the first local feature representation (the system is adapted to perform image pooling by grouping the regions of interest into categories based on identified features using the feature extraction portion and the feature/content recognition models that were trained to identify specific features relating to specific contents; fig 2; paragraphs [0053-0054], [0074]).
As per claim 12, JABER in view of CAMPOS discloses the computer device according to claim 8. JABER fails to disclose wherein the recognizing a category of target content comprised in the target region based on the global feature representation and the second local feature representation comprises: acquiring a content category library, the content category library comprising a preset set of n categories, where n is a positive integer; matching the global feature representation with n categories in the content category library respectively to obtain k candidate categories in the content category library that match the global feature representation, where 0<k<n and k is an integer; performing category sorting on the k candidate categories based on the second local feature representation to obtain a category sorting result; and obtaining a recognition category corresponding to the target content according to the category sorting result.
CAMPOS discloses wherein the recognizing a category of target content comprised in the target region based on the global feature representation and the second local feature representation comprises (a computing system and method for taking an image creating a representation of that image such as a reconstruction or feature map and recognizing an object/content in the image within a region of interest and determining a category of a predetermined number of categories for the object; abstract figs 1-2; paragraphs [0059]): acquiring a content category library, the content category library comprising a preset set of n categories, where n is a positive integer (the memory of the computing device would serve as the content category library and comprises a predetermined number of classes n and n would be positive more than zero; abstract figs 1-2; paragraphs [0059]; claim 6); matching the global feature representation with n categories in the content category library respectively to obtain k candidate categories in the content category library that match the global feature representation, where 0<k<n and k is an integer (using the recognition model the extracted features would be sorted into the n classes and there would comprise k greater than 0 but less than one because each object is sorted into its nearest class making in this example k’s value always equal to one which meets the claimed range of k values; abstract figs 1-2; paragraphs [0059]; claim 6); performing category sorting on the k candidate categories based on the second local feature representation to obtain a category sorting result (the regions of image comprising objects/content is then sorted by the model in the determined class; abstract figs 1-2; paragraphs [0059]; claim 6); and obtaining a recognition category corresponding to the target content according to the category sorting result (via the display and graphic user interface the class the image feature of the image feature map would be observed and the result would be displayed to users using the display; fig 2; paragraphs [0030-0031], [0048], [0112]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to modify JABER to have obtaining a recognition category corresponding to the target content according to the category sorting result of CAMPOS reference. The Suggestion/motivation for doing so would have been to provide the ability to identify high level image features using low level image features as suggested by paragraph [0048] of CAMPOS. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine CAMPOS with JABER to obtain the invention as specified in claim 12.
As per claim 13, JABER in view of CAMPOS discloses the computer device according to claim 8. Modified JABER further discloses wherein the performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation comprises: inputting the target region into a content recognition model, and outputting the image feature representation, wherein the content recognition model is configured to perform deep feature extraction on the target region (the computing system is adapted to input image content to the computing system which then applies a variety of recognition models to determine an image representation 121 and further extract key points and regions of interest; fig 2; paragraphs [0053-0054], [0074-0078]); and performing GeM on the image feature representation to obtain the global feature representation (the computing system is adapted to cluster features together of an image region the clusters can first be identified, and then analyze the saliency measures of the FAST features within each cluster to aid during classification, in these situations, a low-average saliency measure is a number indicating that a corresponding feature is not likely to be very useful in the analysis of a cluster can be an indication of a 3D object while a relatively high-average saliency measure or a large number indicating the corresponding feature is likely to be useful in the analysis of a cluster can indicate a region of text; paragraph [0065]).
As per claim 14, JABER in view of CAMPOS discloses the computer device according to claim 8. Modified JABER further discloses wherein the performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation comprises: performing mean-pooling on the image feature representation to obtain a first global feature representation (the computing system is adapted to use averaging during utilization of a salience algorithm to determine clusters proper category; paragraph [0065]); performing max-pooling on the image feature representation to obtain a second global feature representation (the computing system defines the clustering criteria and the selection criteria 125 could include a low density threshold possibly representing a minimum density (i.e., the minimum density necessary for a particular recognition module to be effective or to be preferable over other recognition modules), high density threshold possibly representing a maximum density (i.e., the maximum density for which a particular recognition module is considered to be effective or preferable over other recognition modules), and/or a feature density range applicable for each recognition module A-N (i.e., the feature density range between a minimum and maximum in which a particular recognition module is deemed most effective and/or preferred over other available recognition modules; paragraphs [0073]); performing GeM on the image feature representation to obtain a third global feature representation (the computing system is adapted to cluster features together of an image region the clusters can first be identified, and then analyze the saliency measures of the FAST features within each cluster to aid during classification, in these situations, a low-average saliency measure (e.g., a number indicating that a corresponding feature is not likely to be very useful in the analysis) of a cluster can be an indication of a 3D object while a relatively high-average saliency measure (e.g., a large number indicating the corresponding feature is likely to be useful in the analysis) of a cluster can indicate a region of text; paragraph [0065]); and performing feature concatenation on the first global feature representation, the second global feature representation, and the third global feature representation to obtain the global feature representation (feature concatenation occurs during the various model training and setting specific thresholds for features that lead to specific outcomes or content types to be sorted in the cluster/category assigned to those feature values this occurs because the same model is adapted to recognize the same features grouping/clustering image regions having the same features together; fig 2; paragraphs [0053-0054], [0074-0078]).
As per claim 15, JABER discloses a non-transitory computer-readable storage medium storing at least one program, the at least one program being loaded and executed by a processor of a computer device and causing the computer device to implement a content recognition method including (a computing system comprising a processor and memory adapted to perform an image processing method directed towards recognizing specific content or activity in image/video frames; abstract; fig 1; paragraphs [0040], [0047-0051]; claim 1): acquiring an image (the computing system is adapted to receive an input image; fig 9; paragraphs [0103], [0108]); recognizing a target region from the image by performing saliency detection on the image (target regions are identified in the input image using saliency algorithms and edge detection features among others; paragraphs [0043], [0050-0052], [0103], [0108]); extracting a key point feature representation corresponding to the image key points in the target region (key points such as image edges and others are extracted using the feature extraction algorithms in order to identify edged and other key points in the images and image regions; fig 2; paragraphs [0053-0054]); performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation (the system is adapted to perform image pooling by grouping the regions of interest into categories based on identified features using the feature extraction portion and the feature/content recognition models that were trained to identify specific features relating to specific contents; fig 2; paragraphs [0053-0054], [0074]); and recognizing a category of target content comprised in the target region based on the global feature representation and the second local feature representation (the computing system is able to recognize content using anyone of models A-x and the models are adapted to sort the image content by content type based on preselected feature values and thresholds learned by the model; fig 2; paragraphs [0053-0054], [0074-0078]). JABER fails to disclose extracting an image feature representation corresponding to the target region; extracting image key points from the target region based on a distribution rule of pixel points in the target region; down sampling the image feature representation based on the target region to obtain a first local feature representation; performing feature concatenation on the key point feature representation and the first local feature representation to obtain a second local feature representation, wherein a size of the second local feature representation is a sum of that of the key point feature representation and the first local feature representation.
CAMPOS discloses extracting an image feature representation corresponding to the target region (taking input images and extracting feature containing patches by labeling them yes or no and these patches act as target regions; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]); extracting image key points from the target region based on a distribution rule of pixel points in the target region (the system extracts sets of low level features called descriptors are extracted from the image without consideration to the relevance information or saliency maps which are updated using the weighted parameters related to the features in the patched image areas; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]); down sampling the image feature representation based on the target region to obtain a first local feature representation (the computing system is adapted to sample images at a variety of sampling scales during training and would be adjusted so the images are down sampled during the described steps; figs 1-2, 4B-5; paragraphs [0034], [0041-0048], [0059], [0080], [0125-0130]); performing feature concatenation on the key point feature representation and the first local feature representation to obtain a second local feature representation, wherein a size of the second local feature representation is a sum of that of the key point feature representation and the first local feature representation (at step 114 the high level extracted features from the image patch region are weighted based on computer generated parameter weighting factors and the applied weights are normalized to sum to a predetermined value such that the values are concatenated as a vector value; figs 1-2, 4B-5; paragraphs [0034], [0041-0048]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to modify JABER to have an image feature representation corresponding to the target region of CAMPOS reference. The Suggestion/motivation for doing so would have been to provide the ability to combine in a single image a set of orderless feature vectors of an input image and produce a weighted image feature map as suggested and provided by figure 4B and paragraph [0049] of CAMPOS. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine CAMPOS with JABER to obtain the invention as specified in claim 15.
As per claim 18, JABER in view of CAMPOS discloses the non-transitory computer-readable storage medium according to claim 15. JABER fails to disclose wherein the recognizing a category of target content comprised in the target region based on the global feature representation and the key point feature representation comprises: acquiring a content category library, the content category library comprising a preset set of n categories, where n is a positive integer; matching the global feature representation with n categories in the content category library respectively to obtain k candidate categories in the content category library that match the global feature representation, where 0<k<n and k is an integer; performing category sorting on the k candidate categories based on the second local feature representation to obtain a category sorting result; and obtaining a recognition category corresponding to the target content according to the category sorting result.
CAMPOS discloses wherein the recognizing a category of target content comprised in the target region based on the global feature representation and the key point feature representation comprises (a computing system and method for taking an image creating a representation of that image such as a reconstruction or feature map and recognizing an object/content in the image within a region of interest and determining a category of a predetermined number of categories for the object; abstract figs 1-2; paragraphs [0059]): acquiring a content category library, the content category library comprising a preset set of n categories, where n is a positive integer (the memory of the computing device would serve as the content category library and comprises a predetermined number of classes n and n would be positive more than zero; abstract figs 1-2; paragraphs [0059]; claim 6); matching the global feature representation with n categories in the content category library respectively to obtain k candidate categories in the content category library that match the global feature representation, where 0<k<n and k is an integer (using the recognition model the extracted features would be sorted into the n classes and there would comprise k greater than 0 but less than one because each object is sorted into its nearest class making in this example k’s value always equal to one which meets the claimed range of k values; abstract figs 1-2; paragraphs [0059]; claim 6); performing category sorting on the k candidate categories based on the second local feature representation to obtain a category sorting result (the regions of image comprising objects/content is then sorted by the model in the determined class; abstract figs 1-2; paragraphs [0059]; claim 6); and obtaining a recognition category corresponding to the target content according to the category sorting result (via the display and graphic user interface the class the image feature of the image feature map would be observed and the result would be displayed to users using the display; fig 2; paragraphs [0030-0031], [0048], [0112]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to modify JABER to have obtaining a recognition category corresponding to the target content according to the category sorting result of CAMPOS reference. The Suggestion/motivation for doing so would have been to provide the ability to identify high level image features using low level image features as suggested by paragraph [0048] of CAMPOS. Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine CAMPOS with JABER to obtain the invention as specified in claim 18.
As per claim 19, JABER in view of CAMPOS discloses the non-transitory computer-readable storage medium according to claim 15. Modified JABER further discloses wherein the performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation comprises: inputting the image into a content recognition model, and outputting the image feature representation, wherein the content recognition model is configured to perform deep feature extraction on the image (the computing system is adapted to input image content to the computing system which then applies a variety of recognition models to determine an image representation 121 and further extract key points and regions of interest; fig 2; paragraphs [0053-0054], [0074-0078]); and performing GeM on the image feature representation to obtain the global feature representation (the computing system is adapted to cluster features together of an image region the clusters can first be identified, and then analyze the saliency measures of the FAST features within each cluster to aid during classification, in these situations, a low-average saliency measure is a number indicating that a corresponding feature is not likely to be very useful in the analysis of a cluster can be an indication of a 3D object while a relatively high-average saliency measure or a large number indicating the corresponding feature is likely to be useful in the analysis of a cluster can indicate a region of text; paragraph [0065]).
As per claim 20, JABER in view of CAMPOS discloses the non-transitory computer-readable storage medium according to claim 15. Modified JABER further discloses wherein the performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation comprises wherein the performing pooling on the image feature representation corresponding to the target region to obtain a global feature representation comprises: performing mean-pooling on the image feature representation to obtain a first global feature representation (the computing system is adapted to use averaging during utilization of a salience algorithm to determine clusters proper category; paragraph [0065]); performing max-pooling on the image feature representation to obtain a second global feature representation (the computing system defines the clustering criteria and the selection criteria 125 could include a low density threshold possibly representing a minimum density (i.e., the minimum density necessary for a particular recognition module to be effective or to be preferable over other recognition modules), high density threshold possibly representing a maximum density (i.e., the maximum density for which a particular recognition module is considered to be effective or preferable over other recognition modules), and/or a feature density range applicable for each recognition module A-N (i.e., the feature density range between a minimum and maximum in which a particular recognition module is deemed most effective and/or preferred over other available recognition modules; paragraphs [0073]); performing GeM on the image feature representation to obtain a third global feature representation (the computing system is adapted to cluster features together of an image region the clusters can first be identified, and then analyze the saliency measures of the FAST features within each cluster to aid during classification, in these situations, a low-average saliency measure (e.g., a number indicating that a corresponding feature is not likely to be very useful in the analysis) of a cluster can be an indication of a 3D object while a relatively high-average saliency measure (e.g., a large number indicating the corresponding feature is likely to be useful in the analysis) of a cluster can indicate a region of text; paragraph [0065]); and performing feature concatenation on the first global feature representation, the second global feature representation, and the third global feature representation to obtain the global feature representation (feature concatenation occurs during the various model training and setting specific thresholds for features that lead to specific outcomes or content types to be sorted in the cluster/category assigned to those feature values this occurs because the same model is adapted to recognize the same features grouping/clustering image regions having the same features together; fig 2; paragraphs [0053-0054], [0074-0078]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Examiner's Note: Examiner has cited figures, and paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested for the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Examiner has also cited references in PTO892 but not relied on, which are relevant and pertinent to the applicant’s disclosure, and may also be reading (anticipatory/obvious) on the claims and claimed limitations. Applicant is advised to consider the references in preparing the response/amendments in-order to expedite the prosecution.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEVIN JACOB DHOOGE whose telephone number is (571) 270-0999. The examiner can normally be reached 7:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800- 786-9199 (IN USA OR CANADA) or 571-272-1000.
/D J DHOOGE/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights