DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/21/2024 have been entered and considered. Initialed copies of the PTO-1449 by the Examiner are attached.
Claim Objections
Claims 2-7, 9-14 and 16-20 are objected to because of the following informalities:
Please uncapitalized the word “claim” in each respective claim. For example, in claim 2, the phrase “The system of Claim 1” should read “The system of Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7-9 and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Checka et al (Pub No.: US20160350336A1) in view of Perronnin et al (Pub No.: US 20090231355).
Regarding independent claim 1, Checka teaches a system for generating and training a color-coding neural network model (methods and system of the present disclosure may apply an auto-encoder architecture to the convolutional neural network representation rather than pixel intensities – see [p][0046] ), comprising: an image processing system (computer system - see Fig 17) comprising a server (multiple interconnected computer systems; e.g., via “the cloud” – see Fig 17 and [p][0083]) and configured to: generate the color-coding neural network (auto-encoder architecture which used descriptor such as color - see [p][0046][0053]) model by: generating layers comprising a visible layer (feature layer – see Fig 1), one or more hidden layers (a convolutional neural network by adding a hidden layer – see [p][0047]) and an output layer (classification layer – see Fig 1); and adding one or more connections to link the visible layer, the one or more hidden layers and the output layer (casting fully connected layers to fully convolutional layers to facilitate generation of a classification map for larger inputs instead of one classification result for the whole image and Fig 1 shows how layer are inter connected – see [p][0034] and Fig 1), wherein each of the one or more connections is defined by parameter matrices; and train the color-coding neural network model (the model is trained to represent an input image with binary codes, which may then be used in classification and visual search – see [p][0047])) by: encoding all parameters of the color-coding neural network model into a single parameter (learn a mapping of images to binary codes. This can be learned within a convolutional neural network by adding a hidden layer that is forced to output 0 or 1 by a sigmoid activation layer, before the classification layer. In this approach the model is trained to represent an input image with binary codes, which may then be used in classification and visual search – see [p][0047]);
Checka does not explicitly teach learning the single parameter by maximizing a log-likelihood function using one or more training images.
However, Perronnin explicitly teaches learning the single parameter by maximizing a log-likelihood function using one or more training images (the set of training pixels in the color space of choice… the parameters of the GMM are suitably estimated by maximizing a log-likelihood function log p(X|.lamda..sup.u) – see [p][0023]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Checka of system for generating and training a color-coding neural network model, with the teachings of Perronnin learning the single parameter by maximizing a log-likelihood function using one or more training images.
Wherein having Checka learning the single parameter by maximizing a log-likelihood function using one or more training images.
. The motivation behind the modification would have been for creating a classifier built for categorizes each category of interest which can adjust at least some pixels of an input image to generate adjusted pixels that are statistically represented by a reference palette since both Checka and Perronnin relates to image classification, wherein Checka processing the image data to select a first subset of the image descriptors that represent a plurality of visual characteristics of the primary image while Perronnin adjusts at least some pixels of an input image to generate adjusted pixels that are statistically represented by a reference palette defined by a mixture model in which each mixture model component is representative of a region of a color space (Please see Checka et al (Pub No.: US20160350336A1) see [p][0048] and Perronnin et al (Pub No.: US 20090231355), see [p][0008]).
Regarding claim 2, Checka in view of Perronnin teach the system of claim 1, Checka teaches wherein the output layer comprises a vector of one or more binary random variables setting a maximum number of possible output clusters (extracted output may be a 4096 dimensional feature vector representing the image and may serve as a basis for the image analysis -see [p][0036]).
Regarding claim 7, Checka in view of Perronnin teach the system of claim 1, Checka teaches wherein two or more output variables of the color-coding neural network model are disjoint from each other ([o]utput from filters in the last convolutional layer may be weighted with trained class specific weights between the following pooling and classification layers to generate activation maps for a particular class -see [p][0038]).
Regarding independent claim 8, Checka teaches method for generating and training a color-coding neural network model: (methods and system of the present disclosure may apply an auto-encoder architecture to the convolutional neural network representation rather than pixel intensities – see [p][0046]), comprising: generating by an image processing system (computer system - see Fig 17) comprising a server (multiple interconnected computer systems; e.g., via “the cloud” – see Fig 17 and [p][0083]) and configured to: generating the color-coding neural network (auto-encoder architecture which used descriptor such as color - see [p][0046][0053]) model by: generating layers comprising a visible layer (feature layer – see Fig 1), one or more hidden layers (a convolutional neural network by adding a hidden layer – see [p][0047]) and an output layer (classification layer – see Fig 1); and adding one or more connections to link the visible layer, the one or more hidden layers and the output layer (casting fully connected layers to fully convolutional layers to facilitate generation of a classification map for larger inputs instead of one classification result for the whole image and Fig 1 shows how layer are inter connected – see [p][0034] and Fig 1), wherein each of the one or more connections is defined by parameter matrices; and train the color-coding neural network model (the model is trained to represent an input image with binary codes, which may then be used in classification and visual search – see [p][0047])) by: encoding all parameters of the color-coding neural network model into a single parameter (learn a mapping of images to binary codes. This can be learned within a convolutional neural network by adding a hidden layer that is forced to output 0 or 1 by a sigmoid activation layer, before the classification layer. In this approach the model is trained to represent an input image with binary codes, which may then be used in classification and visual search – see [p][0047]);
Checka does not explicitly teach learning the single parameter by maximizing a log-likelihood function using one or more training images.
However, Perronnin explicitly teaches learning the single parameter by maximizing a log-likelihood function using one or more training images (the set of training pixels in the color space of choice… the parameters of the GMM are suitably estimated by maximizing a log-likelihood function log p(X|.lamda..sup.u) – see [p][0023]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Checka of system for generating and training a color-coding neural network model, with the teachings of Perronnin learning the single parameter by maximizing a log-likelihood function using one or more training images.
Wherein having Checka learning the single parameter by maximizing a log-likelihood function using one or more training images.
. The motivation behind the modification would have been for creating a classifier built for categorizes each category of interest which can adjust at least some pixels of an input image to generate adjusted pixels that are statistically represented by a reference palette since both Checka and Perronnin relates to image classification, wherein Checka processing the image data to select a first subset of the image descriptors that represent a plurality of visual characteristics of the primary image while Perronnin adjusts at least some pixels of an input image to generate adjusted pixels that are statistically represented by a reference palette defined by a mixture model in which each mixture model component is representative of a region of a color space (Please see Checka et al (Pub No.: US20160350336A1) see [p][0048] and Perronnin et al (Pub No.: US 20090231355), see [p][0008]).
Regarding claim 9, which corresponds to claim 2 except for reciting a different statutory category of a method. Therefore, the rejection analysis of claim 2 is fully applicable to claim 9.
Regarding claim 14, which corresponds to claim 7 except for reciting a different statutory category of a method. Therefore, the rejection analysis of claim 2 is fully applicable to claim 14.
Regarding independent claim 15, Checka teaches a non-tangible computer-readable medium (non-transitory computer-readable medium in signal communication with the processing system – see [p][0008]) embodied with software ([t]he memory 1704 is configured to store software (e.g., program instructions) for execution by the processing system 1702, which software execution – see [p][0084]) for generating and training a color-coding neural network model, the software when executed (methods and system of the present disclosure may apply an auto-encoder architecture to the convolutional neural network representation rather than pixel intensities – see [p][0046]), the software when executed: generates the color-coding neural network model by generating the color-coding neural network (auto-encoder architecture which used descriptor such as color - see [p][0046][0053]) model by: generating layers comprising a visible layer (feature layer – see Fig 1), one or more hidden layers (a convolutional neural network by adding a hidden layer – see [p][0047]) and an output layer (classification layer – see Fig 1); and adding one or more connections to link the visible layer, the one or more hidden layers and the output layer (casting fully connected layers to fully convolutional layers to facilitate generation of a classification map for larger inputs instead of one classification result for the whole image and Fig 1 shows how layer are inter connected – see [p][0034] and Fig 1), wherein each of the one or more connections is defined by parameter matrices; and train the color-coding neural network model (the model is trained to represent an input image with binary codes, which may then be used in classification and visual search – see [p][0047])) by: encoding all parameters of the color-coding neural network model into a single parameter (learn a mapping of images to binary codes. This can be learned within a convolutional neural network by adding a hidden layer that is forced to output 0 or 1 by a sigmoid activation layer, before the classification layer. In this approach the model is trained to represent an input image with binary codes, which may then be used in classification and visual search – see [p][0047]);
Checka does not explicitly teach learning the single parameter by maximizing a log-likelihood function using one or more training images.
However, Perronnin explicitly teaches learning the single parameter by maximizing a log-likelihood function using one or more training images (the set of training pixels in the color space of choice… the parameters of the GMM are suitably estimated by maximizing a log-likelihood function log p(X|.lamda..sup.u) – see [p][0023]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Checka of system for generating and training a color-coding neural network model, with the teachings of Perronnin learning the single parameter by maximizing a log-likelihood function using one or more training images.
Wherein having Checka learning the single parameter by maximizing a log-likelihood function using one or more training images.
. The motivation behind the modification would have been for creating a classifier built for categorizes each category of interest which can adjust at least some pixels of an input image to generate adjusted pixels that are statistically represented by a reference palette since both Checka and Perronnin relates to image classification, wherein Checka processing the image data to select a first subset of the image descriptors that represent a plurality of visual characteristics of the primary image while Perronnin adjusts at least some pixels of an input image to generate adjusted pixels that are statistically represented by a reference palette defined by a mixture model in which each mixture model component is representative of a region of a color space (Please see Checka et al (Pub No.: US20160350336A1) see [p][0048] and Perronnin et al (Pub No.: US 20090231355), see [p][0008]).
Regarding claim 16, which corresponds to claim 2 except for reciting a different statutory category of a non-tangible computer-readable. Therefore, the rejection analysis of claim 2 is fully applicable to claim 16.
Claims 3, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Checka et al (Pub No.: US20160350336A1) in view of Perronnin et al (Pub No.: US 20090231355) as applied to claims 1, 8 and 15 further in view of Bhardwaj et al (NPL titled: A genetically optimized neural network for classification of breast cancer disease).
Regarding claim 3, Checka in view of Perronnin does not explicitly teach the system of Claim 1, wherein each of the one or more hidden layers comprises one or more floating point random variable nodes.
However, Bhardwaj explicitly teaches wherein each of the one or more hidden layers comprises one or more floating point random variable nodes ([t]he only thing allowed below a weighting function W on the left child is either a floating-point random constant or an arithmetic function – see section II, subsection A, [p][004]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Checka as modified by Perronnin of system for generating and training a color-coding neural network model, with the teachings of Bhardwaj wherein each of the one or more hidden layers comprises one or more floating point random variable nodes
Wherein having Checka wherein each of the one or more hidden layers comprises one or more floating point random variable nodes.
. The motivation behind the modification would have been for creating a classifier built for categorizes each category of interest which give an optimal structure for classification since both Checka and Bhardwaj relates to image classification, wherein Checka processing the image data to select a first subset of the image descriptors that represent a plurality of visual characteristics of the primary image while Perronnin give an optimal structure for classification which helps the algorithm to reach solution faster (Please see Checka et al (Pub No.: US20160350336A1) see [p][0048] and Bhardwaj et al (NPL titled: A genetically optimized neural network for classification of breast cancer disease), see section IV]).
Regarding claim 10, which corresponds to claim 3 except for reciting a different statutory category of a method. Therefore, the rejection analysis of claim 3 is fully applicable to claim 10.
Regarding claim 17, which corresponds to claim 3 except for reciting a different statutory category of a non-tangible computer-readable. Therefore, the rejection analysis of claim 3 is fully applicable to claim 17.
Claims 4, 11 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Checka et al (Pub No.: US20160350336A1) in view of Perronnin et al (Pub No.: US 20090231355) as applied to claims 1, 8 and 15 further in view of Munawar (Pub No.: 20170076224)
Regarding claim 4, Checka in view of Perronnin does not explicitly teach the system of Claim 1, wherein training the color-coding neural network model further comprises: applying a layer-to-layer contrastive divergence algorithm.
However, Munawar explicitly teaches wherein training the color-coding neural network model further comprises: applying a layer-to-layer contrastive divergence algorithm (the positive training module 132 performs an ordinary contrastive divergence method to adjust the parameters of the autoencoder – see [p][0072]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Checka as modified by Perronnin of system for generating and training a color-coding neural network model, with the teachings of Munawar wherein training the color-coding neural network model further comprises: applying a layer-to-layer contrastive divergence algorithm.
Wherein having Checka wherein training the color-coding neural network model further comprises: applying a layer-to-layer contrastive divergence algorithm.
. The motivation behind the modification would have been for creating a classifier built for categorizes each category of interest and updates the parameters based on the calculated changes in a normal manner since both Checka and Munawar relates to image classification, wherein Checka processing the image data to select a first subset of the image descriptors that represent a plurality of visual characteristics of the primary image while Munawar adjust the parameters of the autoencoder by updating the parameters based on the calculated changes in a normal manner (Please see Checka et al (Pub No.: US20160350336A1) see [p][0048] and Munawar et al (Pub No.: 20170076224), see [p][0072]).
Regarding claim 11, which corresponds to claim 4 except for reciting a different statutory category of a method. Therefore, the rejection analysis of claim 4 is fully applicable to claim 11.
Regarding claim 18, which corresponds to claim 4 except for reciting a different statutory category of a non-tangible computer-readable. Therefore, the rejection analysis of claim 4 is fully applicable to claim 18.
Claims 5, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Checka et al (Pub No.: US20160350336A1) in view of Perronnin et al (Pub No.: US 20090231355) as applied to claims 1, 8 and 15 further in view of Cote et al (NPL titled: An Infinite Restricted Boltzmann Machine).
Regarding claim 5, Checka in view of Perronnin does not explicitly teach the system of claim 1, wherein the color-coding neural network model comprises a joint probability distribution modelled using a visible variable, a hidden variable, an output variable, a partition function and an energy function.
However, Cote explicitly teaches wherein the color-coding neural network model comprises a joint probability distribution modelled (joint probability – see section [p][005], equation 15 ) using a visible variable (v, visible variable - see section [p][005], equation 15), a hidden variable (v, visible variable - see section [p][005], equation 15), an output variable, a partition function (z, partition variable - see section [p][005], equation 15) and an energy function (derive a formulation where the free energy depends only on v - see section [p][005], equation 15).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Checka as modified by Perronnin of system for generating and training a color-coding neural network model, with the teachings of Cote wherein the color-coding neural network model comprises a joint probability distribution modelled using a visible variable, a hidden variable, an output variable, a partition function and an energy function.
Wherein having Checka wherein the color-coding neural network model comprises a joint probability distribution modelled using a visible variable, a hidden variable, an output variable, a partition function and an energy function.
. The motivation behind the modification would have been for creating a classifier built for categorizes each category of interest for naturally and adaptively adds trained hidden units during learning since both Checka and Cote relates to image classification, wherein Checka processing the image data to select a first subset of the image descriptors that represent a plurality of visual characteristics of the primary image while Cote naturally and adaptively adds trained hidden units during learning (Please see Checka et al (Pub No.: US20160350336A1), see [p][0048] and Cote et al (NPL titled: An Infinite Restricted Boltzmann Machine), see Abstract).
Regarding claim 12, which corresponds to claim 5 except for reciting a different statutory category of a method. Therefore, the rejection analysis of claim 5 is fully applicable to claim 12.
Regarding claim 19, which corresponds to claim 5 except for reciting a different statutory category of a non-tangible computer-readable. Therefore, the rejection analysis of claim 5 is fully applicable to claim 19.
Claims 6, 13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Checka et al (Pub No.: US20160350336A1) in view of Perronnin et al (Pub No.: US 20090231355) as applied to claims 1, 8 and 15 further in view of Larlus-Larrondo et al (Pub No.: 20160307071).
Regarding claim 6, Checka in view of Perronnin does not explicitly teach the system of claim 1, wherein an input variable of the color-coding neural network model is modeled as a Gaussian with a diagonal covariance.
However, Larlus-Larrondo explicitly teaches wherein an input variable of the color-coding neural network model is modeled as a Gaussian with a diagonal covariance (deviation vector of Gaussian k (assuming a diagonal covariance) – see [p][0029]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Checka as modified by Perronnin of system for generating and training a color-coding neural network model, with the teachings of Larlus-Larrondo wherein training the color-coding neural network model further comprises: applying a layer-to-layer contrastive divergence algorithm.
Wherein having Checka wherein training the color-coding neural network model further comprises: applying a layer-to-layer contrastive divergence algorithm.
. The motivation behind the modification would have been for creating a classifier built for categorizes each category of interest and generating a training image feature vector representing the training image since both Checka and Larlus-Larrondo relates to image classification, wherein Checka processing the image data to select a first subset of the image descriptors that represent a plurality of visual characteristics of the primary image while Larlus-Larrondo generating a training image feature vector representing the training image (Please see Checka et al (Pub No.: US20160350336A1) see [p][0048] and Larlus-Larrondo et al (Pub No.: 20160307071), see [p][0007]).
Regarding claim 13, which corresponds to claim 6 except for reciting a different statutory category of a method. Therefore, the rejection analysis of claim 6 is fully applicable to claim 13.
Regarding claim 20, which corresponds to claim 6 except for reciting a different statutory category of a non-tangible computer-readable. Therefore, the rejection analysis of claim is fully applicable to claim 20.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Amer et al (Pub No.: 20160307071) discloses method for analyzing multi-task multimodal data to detect multi-task multimodal events using a deep multi-task representation learning, are disclosed. A combined model with both generative and discriminative aspects is used to share information during both generative and discriminative processes. The technologies can be used to classify data and also to generate data from classification events. The data can then be used to morph data into a desired classification event.
Colson et al (Pub No.: 20160292769) discloses a system and methods described herein employ adaptive machine learning to provide recommendations to an entity that selects one or more items for a client from an item inventory. Client information, item information, and recommendation algorithms are stored and are accessible by a recommendation engine. The recommendation algorithms utilize the client information and the item information in different manners to identify different subsets of items recommended for a client. Information about two or more of the subsets of the items in the item inventory that are identified are selected for display to the entity tasked with selecting items for the client. Feedback information, including client, selection and/or coverage feedback information, is obtained and adaptive machine learning is used to modify the stored client information, the stored item information and/or stored recommendation algorithm(s), in dependence on the client feedback information, the selection feedback information and/or the coverage feedback information.
Shimozaki et al (Pub No.: US20230122261A1) teaches methods, systems, and non-transitory computer-readable medium for color and pattern analysis of images including wearable items. For example, a method may include receiving an image depicting a wearable item, identifying the wearable item within the image by identifying a face of an individual wearing the wearable item or segmenting a foreground silhouette of the wearable item from background image portions of the image, determining a portion of the wearable item identified within the image as being a patch portion representative of the wearable item depicted within the image, deriving one or more patterns of the wearable item based on image analysis of the determined patch portion of the image, deriving one or more colors of the wearable item based on image analysis of the determined patch portion of the image, and transmitting information regarding the derived one or more colors and information regarding the derived one or more patterns.
Cornebise et al (Pub No.: 20140279777 ) discloses a signal processor, the signal processor comprising: a probability vector generation system, wherein said probability vector generation system has an input to receive a category vector for a category of output example and an output to provide a probability vector for said category of output example, wherein said output example comprises a set of data points, and wherein said probability vector defines a probability of each of said set of data points for said category of output example; a memory storing a plurality of said category vectors, one for each of a plurality of said categories of output example; and a stochastic selector to select a said stored category of output example for presentation of the corresponding category vector to said probability vector generation system; wherein said signal processor is configured to output data for an output example corresponding to said selected stored category.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDRAE S ALLISON whose telephone number is (571)270-1052. The examiner can normally be reached on Monday-Friday 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns, can be reached on (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDRAE S ALLISON/Primary Examiner, Art Unit 2673
April 17, 2026
Prosecution Insights