DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
(Submitted on 6/3/2026)
In regard to 103 rejections
- The applicant has amended the independent claims 1, 12, 21 and 22. The applicant has provided no
other amendments. On Page 1, the applicant argues on the prior art “Lau” and “Broyda” with respect to amended limitations specific to “ balancing the training data across plurality of camera types”. The applicant specifically argues that the reference “Broyda” does not teach different camera types with respect to different make and model of the camera and types of camera not having different image capture characteristics.
Examiner’s Response
There is no specifics in the claim limitation to this and as such the different types of cameras may be interpreted with different context perhaps as known to a POSITA. Further to substantiate the new limitations focusing on the “camera”, another four new references ““Shigeta” , “Pouyan”, “Perna” and “Krish”. Reference “Shigeta” teaches different types of cameras for claim 12 and also teaches within the context of printing Manufacturing code, the different types of cameras. Reference “Pouyan” teaches different types of cameras (network cameras) for claims 1, and 21 . Reference “Perna” teaches the claim 22. Reference ” Krish” teaches the amendments for claims 1, 12, 21 and 22 for balancing the training set. The examiner submits that surveillance network cameras as taught by the reference “ Pouyan” can be integrated into product authentication systems when their secure, networked capabilities are used to verify the legitimacy of goods, track their movement, and prevent counterfeiting.
In CONCLUSION, the examiner rejects the independent claims 1, 12, 21 and 22 and all dependent claims 3-8, 10, 14-20 and 23-24 and MOVE the applicant as FINAL REJECTION under 103.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 8, 10, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over
Tak Wai Lau et. al (hereinafter Lau) US 2020/0410510 A1,
in view of Samira Pouyanfar et.al (hereinafter Pouyan), Dynamic Sampling in Convolutional Neural Networks for Imbalanced Data Classification, 2018 IEEE Conference on Multimedia Information Processing and Retrieval.
in view of Madhusudan Krishnamoorthy et.al (hereinafter Krish) US 2020/0218825 A1.
In regard to claim 1: (Currently Amended)
Lau discloses:
- A machine learning based imaging method for imaging and classifying whether one or more physical and subject consumer goods are authentic or non-authentic, the machine learning based imaging method comprising:
[Abstract]:
An authentication apparatus and a method to devise an authentication tool is provided for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of an information bearing device on the article. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device using a trained neural network,
- a) obtaining an image of a subject consumer good comprising a subject product specification
[0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed,
[0070]:
A direct image of a source means the image is obtained directly from the source without intervening copying, that is, the image is not captured from an image of the source. An example source may be an information bearing device which is covertly coded with a security data designed to function as an authentic authentication device,
- b) inputting the obtained image into a model, wherein the model is configured to classify the obtained image as authentic or non- authentic
[0102]:
During the forward pass on progressing from an earlier layer to a next layer, each filter is convolved across the spatial dimensions of the input volume,
[0102]:
The entries of the filter collectively define a weight matrix and the weight matrix was learned by the CNN during deep learning training of the CNN,
- wherein the model is constructed by a machine learning classifier,
[0075]:
An example CNN 30 comprises an input layer 300, an output layer 399, and a plurality of convolutional layers 301-30n interconnecting the input layer 300 and the output layer 399. CNN is a class of deep, feed-forward, artificial neural networks in machine learning,
- of an authentic product comprising an authentic product specification comparable with the subject product specification
[0187]:
The training images include images of authentic and non-authentic information bearing devices,
[0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed. To facilitate verification of authenticity of an article, an article is commonly incorporated with an authentication device which includes an information bearing device such as a label, a tag or an imprint. The information bearing device comprises a data-embedded image pattern and the data-embedded image pattern is covertly encoded with a set of data so that the data is not perceivable by a reasonable person reading the data-embedded image pattern
- wherein the authentic product specification comprises at least one steganographic feature having a length greater than 0.01 mm;
[0108]:
the example information bearing device 60 is set to have an physical size of a one-cm square,
[0109]:
To print the information bearing device using a 1200 DPI printer on a 1 cm× 1 cm medium, the information bearing device 60 need to be resized and quantized. Specifically, the information bearing device is needed to resize to a width and height of 472 pixels in each orthogonal direction, since 472 pixels per cm is equivalent to 1200 DPI. Each pixel of the data-embedded image pattern is a real number and the resized information bearing device is quantized from real number to bi-level,
[0070]:
An example source may be an information bearing device which is covertly coded with a security data designed to function as an authentic authentication device,
[0070] :
The covertly coded data is typically not human readable or perceivable and the data coding may be by means of steganographic techniques such as transform domain coding techniques.
- and (ii) an associated class definition based on the steganographic feature;
[0080]:
An example CNN of an example authentication apparatus comprises a plurality of convolution layers between the input layer and the output layer, as depicted in FIG. 3. The convolution layers of the CNN are serially connected to form an ensemble of serially connected convolution layers. Each convolution layer comprises a plurality of filters, and each filter is a convolution filter which is to operate with an input data file to generate an output data file. A plurality of output data files is generated as a result of convolution operations among the convolution filters and the input data files at the input of a convolution layer. Each output data file is referred to as a feature map in CNN terminology.
[0079]:
The fully connected network (“FCN”) is connected to output of the CNN, such that output of the CNN is fed as input to the FCN, as depicted in FIG. 2. The FCN will perform classification operations on the processed data of the CNN, for example, to determine whether, or how likely, the processed data of a target image CNN corresponds to an authentic authentication device or a non-authentic authentication device.
- c) outputting a classification output from the model indicating a likelihood that the image of the subject consumer good is authentic or non-authentic
[0079]:
The FCN will perform classification operations on the processed data of the CNN, for example, to determine whether, or how likely, the processed data of a target image CNN corresponds to an authentic authentication device or a non-authentic authentication device.
- wherein the non-authentic product specification is different from the at least one steganographic feature [9]
[0073]:
An authentic authentication device herein is also referred to as an authentic information bearing device or a genuine information bearing device herein, while a non-authentic authentication device is also referred to as a non-authentic information bearing device or a non-genuine information bearing device where appropriate. The target image may be captured by the apparatus or received from an outside source.
[0132]:
An image of an authentic authentication device is a primary copy of an authentic information bearing device, while an image of a non-authentic authentication device may be a secondary copy of an authentic information bearing device or a copy of a fake information bearing device.
[0071]:
An example information bearing device herein comprises a data-encoded image pattern which is encoded with a set of discrete data. The set of data is human non-perceivable in its encoded state such that the data is not readily readable or readily decodable by a human reader looking at the data-encoded image pattern using naked eyes.
- and wherein the training dataset further comprises the extracted images of the authentic product augmented with geometric distortion so that the extracted images of the authentic product have a different shape. [10]
[0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed. To facilitate verification of authenticity of an article, an article is commonly incorporated with an authentication device which includes an information bearing device such as a label, a tag or an imprint. The information bearing device comprises a data-embedded image pattern and the data-embedded image pattern is covertly encoded with a set of data so that the data is not perceivable by a reasonable person reading the data-embedded image pattern. In example embodiments. Each data is a discrete data having characteristic two- or three-dimensional coordinate values in data domain and the coordinate values are transformed into spatial properties of image-defining elements which cooperate to define the entirety of the data-embedded image pattern. The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
(BRI: transforming data into spatial properties of image-defining elements (such as pixel positions, shapes, or sizes) and spreading coordinate values throughout those elements generally represents geometrical distortion)
[0016] :
In some embodiments, the set of data embedded in the data-embedded image pattern comprises a plurality of discrete frequency data, and the discrete frequency data are transformed into spatially distributed pattern defining elements which are spread in the data-embedded image pattern and which are non-human readable or non-human perceivable using naked eyes; and the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
[0136]:
Each pixel has characteristic physical properties including size, shape, color, brightness, etc., and the entirety of pixels collectively define a data-embedded image pattern.
(BRI: the process of spatially distributing a pattern that are non-human readable represents a sophisticated steganographic feature)
- wherein the training dataset is spatially manipulated by a Spatial Transformer Network before training the machine learning classifier.
[0006]:
The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
[0016]:
the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
[0177]:
On defining the CNN structure, the input layer is set to have a single channel since the example information bearing device 60 has a data-embedded image pattern which is defined by pattern defining elements in gray-scale coding.
(BRI: this process that combine frequency domain data with spatial manipulation, such as those used in Spectral-Spatial-Frequency Transformer Networks or Fourier-based data augmentation/feature extraction. Specifically, discrete frequency data (e.g., Fourier or Discrete Cosine Transform coefficients) can be transformed back into spatially distributed patterns or maps, which are then used to augment or inform training datasets. When these generated patterns are treated as input images, a Spatial Transformer Network (STN) can be employed to actively manipulate (e.g., rotate, scale, warp) these input features to improve spatial invariance before the data is fed to a classification model.
Lau does not explicitly disclose:
- wherein the machine learning classifier is trained by a training dataset, wherein the training dataset comprises : i) extracted images, from a plurality of different camera types,
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics
However, Pouyan discloses:
- wherein the machine learning classifier is trained by a training dataset, wherein the training dataset comprises : i) extracted images, from a plurality of different camera types,
[3.5.2, Page 115]:
The process for finding the network cameras depends on the types of the cameras. Some cameras have built-in web servers to distribute the data and each camera has a unique IP address.
[BRI: a network camera can be used for product authentication deployed with the right technology and integration that captures images of products at key points in the supply chain (e.g., on a production line, at a retail counter, or in a warehouse]
[4.1, Page 115]:
retrieve an image from every active camera in
[4.1, Page 116]:
the database.
a check program traverse the directory to remove the bad quality image based on their byte sizes. The final cleaned dataset contains over 10,000 images captured from network cameras.
[ 4.2, Page 116]:
CNN model plus a modified data augmentation in which each training batch includes a balanced number of classes
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics
[3.5.2, Page 115]:
Although many network cameras provide data to the public, finding them is not always easy because of the wide variety of brands and models. The data from many network cam eras are aggregated on web servers, and there are many different ways of organizing the data streams. To use the data from the network cameras, this system has a database of publicly available network cameras. The process for finding the network cameras depends on the types of the cameras.
[BRI: network camera refers to a surveillance network that contains multiple types of cameras, each designed for different environments, coverage needs, and image capture characteristics. In such a setup, you can indeed have three different camera types with distinct image capture characteristics]
[1, Page 112]:
Millions of network cameras (a type of surveillance cameras) have been deployed in city streets, tourist attractions, and many other locations.
[1, Page 112]:
Figure 1 (a)-(d) show example images from network cameras, including roads, buildings, vehicles, and people. Figure 1 (e)-(i) show example images from several popular datasets. In contrast to the most commonly used datasets for classification, localization, ob ject detection, and segmentation, network cameras contain many objects that occupy small portions of an image and are not necessarily at the center. This paper introduces the first version of the network camera image dataset
PNG
media_image1.png
317
537
media_image1.png
Greyscale
[3.5.2, Page 115]:
The process for finding the network cameras depends on the types of the cameras. Some cameras have built-in web servers to distribute the data and each camera has a unique IP address. The data can be accessed by connecting directly to this address and using the HTTP GET requests. Different brands require different GET commands. For example, Axis cameras use /mjpg/video.mjpg but Foscam uses /video.cgi.
[BRI: a network camera can be used for product authentication deployed with the right technology and integration that captures images of products at key points in the supply chain (e.g., on a production line, at a retail counter, or in a warehouse]
[Abstract, Page 112]:
The proposed model can discover the semantic concepts from the data with a skewed distribution using a dynamic sampling technique. The paper also presents a system that can retrieve real-time visual data from heterogeneous cameras, and the run-time
environment allows the analysis programs to process the data from thousands of cameras simultaneously. The evaluation results in comparison with several state-of-the-art methods demonstrate the ability and effectiveness of the proposed model on visual data captured by public network cameras
[3.5.2, Page 115]:
Discovering Heterogeneous Network Cameras Although many network cameras provide data to the public, finding them is not always easy because of the wide variety of brands and models. The data from many network cam eras are aggregated on web servers, and there are many different ways of organizing the data streams.
[4.2, Page 116]:
The F1-score (Avg. F1) is adopted as the main evaluation metric since it is the most valuable comparison metric for imbalanced data
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, and Pouyan.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
One of ordinary skill would have motivation to combine Lau, and Pouyan that can provide improvement to the performance (classification) of the minority classes and maintain performance of the majority classes (Pouyan [1, Page 113]).
Lau and Pouyan do not explicitly disclose:
- wherein the training dataset further comprises further extracted images, from the plurality of different camera types, of a non-authentic product comprising a non-authentic product specification
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types,
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
However, Krish discloses:
- wherein the training dataset further comprises further extracted images, from the plurality of different camera types, of a non-authentic product comprising a non-authentic product specification
[0052]:
In an exemplary embodiment, the one or more advanced machine learning models may determine the authenticity of data by verifying that the data that is provided by a user for predicting the outcome is associated with a distribution that is used in training the one or more advanced models and may produce an outcome
[0059]:
As illustrated in block 610, the system executes instructions in the security module to (1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not. In one embodiment where the one or machine learning models identify that the data is not authentic, the one or more machine learning models do not produce an outcome. In another embodiment where the one or more machine learning models identify that the data is authentic, the one or more machine learning models produce an outcome.
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types
[0040] :
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1 in more detail, in accordance with embodiments of the invention. However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras, video recorders, audio/video player, radio, GPS devices, wearable devices, Internet-of-things devices, augmented reality devices, virtual reality devices, automated teller machine devices, electronic kiosk devices, or any combination of the aforementioned.
[0060]]:
As illustrated in block 615, the system executes instructions in the equality module to (1) provide disparity metrics to the one or more machine learning models, (2) provide bias tolerance levels to the one or more machine learning models, and (3) provide debiasing capabilities to the one or more machine learning models.
[0059]:
(1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
{BRI: by combining GAN-based data augmentation with centroid balancing, represents a robust training across a plurality of camera types.
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
[Abstract]:
Embodiments of the present invention provide a system for providing a centralized advanced security provisioning platform to create reliable machine learning models and also to enhance the existing machine learning models. The system is configured for executing instructions in the privacy module to monitor and control data privacy and data usage, executing instructions in the security module to preserve the authenticity of data that is used by the machine learning models to predict an outcome, executing instructions in the equality module to detect and prevent biasing of the machine learning models,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1
[0040]:
However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras
[0052]:
machine learning models to identify whether the data that is being used by the one or more advanced machine learning models is authentic or not.
[0053]:
one or more advanced machine learning models will allow the one or more advanced machine learning models to identify whether the training data is genuine or not. The one or more advanced machine learning models may identify that the training data is genuine or not by calculating and comparing a centroid of the training data provided by the data scientists with a centroid of the raw training data
[0053]:
The method implemented by the one or more advanced machine learning models to detect whether the training data is genuine or not is centroid balancing as shown in layer 520 of FIG. 5.
[0059]:
(1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
[0060]:
(3) provide debiasing capabilities to the one or more machine learning models.
[0024]:
in a variety of fields and there are many applications that are closely related to our daily life, such as making significant decisions in application area based on predictions or classifications, in which a Machine Learning (ML) model could be relevant. Hence, if a ML model causes mispredictions or misclassifications due to malicious external influences, it can cause catastrophic complications.
[BRI: Perhaps known to the POSITA that misclassifications or mispredictions caused by malicious external influences can absolutely apply to product authenticity. In fact, such attacks are a growing concern in supply chain and verification systems. If attackers can manipulate the training data (data poisoning) or inject adversarial inputs (adversarial attacks), the model’s decision-making can be subtly altered. Attackers inject counterfeit product images or labels into training datasets so the model learns to misclassify genuine items as fake]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan and Krish.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
One of ordinary skill would have motivation to combine Lau, Pouyan and Krish that can provide enriched performance using machine learning in applications area (Krish[0024]) that includes authenticity [0052]).
In regard to claim 3: (Previously Presented)
Lau discloses:
- augmenting the extracted images of the authentic product with color distortion
[0136]:
Modern authentic information bearing devices contain data-embedded image patterns which are digitally formed and consist of pixels. Each pixel has characteristic physical properties including size, shape, color, brightness, etc,
[0136 ]:
Some of the characteristic physical properties suffer degradation or loss of fidelity during image capture and/or reproduction,
[0138]:
When the data-embedded image pattern of the authentic information bearing device is captured by an image capture apparatus, the gray levels of the pixels forming the captured image may be changed. For example, the gray levels may be shifted linearly, non-linearly, randomly or may have an entirely different gray-scale distribution of pixels compared to those of the data-embedded image pattern. The change may be due to internal setting of the image capture apparatus (for example, exposure setting), calibration of the image capture apparatus, ambient illumination, sensitivity and/or linearity of the image sensor of the capture apparatus, angle of image capture, and/or other parameters.
In regard to claim 4: (Original)
Lau discloses:
- the at least one steganographic feature has a length from 0.02 mm to 20 mm
[0109]:
To print the information bearing device using a 1200 DPI printer on a 1 cm × 1 cm medium, the information bearing device 60 need to be resized and quantized. Specifically, the information bearing device is needed to resize to a width and height of 472 pixels in each orthogonal direction, since 472 pixels per cm is equivalent to 1200 DPI. Each pixel of the data-embedded image pattern is a real number and the resized information bearing device is quantized from real number to bi-level,
[0070]:
An example source may be an information bearing device which is covertly coded with a security data designed to function as an authentic authentication device. The authentic authentication device is a target for the purpose of the present disclosure and the source is therefore also a target. The covertly coded data is typically not human readable or perceivable and the data coding may be by means of steganographic techniques such as transform domain coding techniques.
In regard to claim 5: (Original)
Lau discloses:
- the machine learning classifier is validated by a validating dataset, wherein the validating dataset comprises one or more images defining the at least one steganographic feature of the subject product specification
[0006]:
To facilitate verification of authenticity of an article, an article is commonly incorporated with an authentication device which includes an information bearing device such as a label, a tag or an imprint.
In regard to claim 8: (Original)
Lau discloses:
- the machine learning classifier is a convolutional neural network (CNN)
[0008]:
In some embodiments, the neural network is a convolutional neural network (CNN).
In regard to claim 10: (Original)
Lau discloses:
- the obtained image of the subject consumer good is spatially manipulated before being inputted into the model
[0080]:
An input data file presented at the input of a first convolution layer of the CNN is intended to be a data file of a target image containing a plurality of image data representing a plurality of image-defining elements. Each image-defining element has spatial properties and characteristics such that the spatial properties and characteristics of all the image-defining elements of a target image define the entirety of the target image. The spatial properties and characteristics include spatial coordinates and signal amplitude or strength of the image-defining elements.
that includes authenticity [0052]).
In regard to claim 21: (Currently Amended )
Lau discloses:
- imaging and classifying whether one or more physical and subject consumer goods are authentic or non- authentic, that when executed by one or more processors cause the one or more processors to: [0068]:,
in [0006] :
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed.
- a) obtain an image of a subject consumer good comprising a subject product specification
in [0070] :
A direct image of a source means the image is obtained directly from the source without intervening copying, that is, the image is not captured from an image of the source. An example source may be an information bearing device which is covertly coded with a security data designed to function as an authentic authentication device.
(BRI: The product specification is the source information on the information bearing device which is overtly coded)
- b) input the obtained image into a model, wherein the model is configured to classify the obtained image as authentic or non- authentic,
in [0102] :
During the forward pass on progressing from an earlier layer to a next layer, each filter is convolved across the spatial dimensions of the input volume,
in [0102]:
The entries of the filter collectively define a weight matrix and the weight matrix was learned by the CNN during deep learning training of the CNN,
in [0187]:
The training images include images of authentic and non-authentic information bearing devices,
- wherein the model is constructed by a machine learning classifier,
in [0075] :
An example CNN 30 comprises an input layer 300, an output layer 399, and a plurality of convolutional layers 301-30n interconnecting the input layer 300 and the output layer 399. CNN is a class of deep, feed-forward, artificial neural networks in machine learning,
- wherein the machine learning classifier is trained by a training dataset,
[0183] :
The training images are selected according to some selection criteria such that the imperfection values are within the acceptable ranges”,
[0187]:
The training images include images of authentic and non-authentic information bearing devices.
- where in training data set comprises: of an authentic product specification comprising an authentic product specification comparable with the subject product specification
[0072]:
Due to its unique properties, for example, a specific or one-to-one correspondence between a set of data and a set of spatial image pattern having spread or distributed pattern defining elements to represent the set of data,
[0072]:
the information bearing device can be used as an authentication device, with the encoded coordinate data or encoded set of coordinate data,
[0073] :
An authentic authentication device herein is also referred to as an authentic information bearing device or a genuine information bearing device herein, while a non-authentic authentication device is also referred to as a non-authentic information bearing device or a non-genuine information bearing device where appropriate.
- the authentic product specification comprises at least one steganographic feature having a length greater than 0.01 mm;
[0108]:
the example information bearing device 60 is set to have an physical size of a one-cm square,
[0109]:
To print the information bearing device using a 1200 DPI printer on a 1 cm× 1 cm medium, the information bearing device 60 need to be resized and quantized. Specifically, the information bearing device is needed to resize to a width and height of 472 pixels in each orthogonal direction, since 472 pixels per cm is equivalent to 1200 DPI. Each pixel of the data-embedded image pattern is a real number and the resized information bearing device is quantized from real number to bi-level,
[0070] :
An example source may be an information bearing device which is covertly coded with a security :
data designed to function as an authentic authentication device,
[0070] :
The covertly coded data is typically not human readable or perceivable and the data coding may be by means of steganographic techniques such as transform domain coding techniques.
- (ii) an associated class definition based on the steganographic feature;
[0080]:
An example CNN of an example authentication apparatus comprises a plurality of convolution layers between the input layer and the output layer, as depicted in FIG. 3. The convolution layers of the CNN are serially connected to form an ensemble of serially connected convolution layers. Each convolution layer comprises a plurality of filters, and each filter is a convolution filter which is to operate with an input data file to generate an output data file. A plurality of output data files is generated as a result of convolution operations among the convolution filters and the input data files at the input of a convolution layer. Each output data file is referred to as a feature map in CNN terminology.
[0079]:
The fully connected network (“FCN”) is connected to output of the CNN, such that output of the CNN is fed as input to the FCN, as depicted in FIG. 2. The FCN will perform classification operations on the processed data of the CNN, for example, to determine whether, or how likely, the processed data of a target image CNN corresponds to an authentic authentication device or a non-authentic authentication device.
- c) output a classification output from the model indicating a likelihood that the image of the subject consumer good is authentic or non-authentic
[0079]:
The FCN will perform classification operations on the processed data of the CNN, for example, to determine whether, or how likely, the processed data of a target image CNN corresponds to an authentic authentication device or a non-authentic authentication device.
- wherein the training dataset further comprises the extracted images of the authentic product augmented with geometric distortion so that the extracted images of the authentic product have a different shape,
[0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed. To facilitate verification of authenticity of an article, an article is commonly incorporated with an authentication device which includes an information bearing device such as a label, a tag or an imprint. The information bearing device comprises a data-embedded image pattern and the data-embedded image pattern is covertly encoded with a set of data so that the data is not perceivable by a reasonable person reading the data-embedded image pattern. In example embodiments. Each data is a discrete data having characteristic two- or three-dimensional coordinate values in data domain and the coordinate values are transformed into spatial properties of image-defining elements which cooperate to define the entirety of the data-embedded image pattern. The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
[BRI: transforming data into spatial properties of image-defining elements (such as pixel positions, shapes, or sizes) and spreading coordinate values throughout those elements generally represents geometrical distortion)
[0016] :
In some embodiments, the set of data embedded in the data-embedded image pattern comprises a plurality of discrete frequency data, and the discrete frequency data are transformed into spatially distributed pattern defining elements which are spread in the data-embedded image pattern and which are non-human readable or non-human perceivable using naked eyes; and the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
[0136]:
Each pixel has characteristic physical properties including size, shape, color, brightness, etc., and the entirety of pixels collectively define a data-embedded image pattern.
(BRI: the process of spatially distributing a pattern that are non-human readable represents a sophisticated steganographic feature)
- and wherein the training dataset is spatially manipulated by a Spatial Transformer Network before training the machine learning classifier.
[0006]:
The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
[0016]:
the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
[0177]:
On defining the CNN structure, the input layer is set to have a single channel since the example information bearing device 60 has a data-embedded image pattern which is defined by pattern defining elements in gray-scale coding.
[BRI: this process that combine frequency domain data with spatial manipulation, such as those used in Spectral-Spatial-Frequency Transformer Networks or Fourier-based data augmentation/feature extraction. Specifically, discrete frequency data (e.g., Fourier or Discrete Cosine Transform coefficients) can be transformed back into spatially distributed patterns or maps, which are then used to augment or inform training datasets. When these generated patterns are treated as input images, a Spatial Transformer Network (STN) can be employed to actively manipulate (e.g., rotate, scale, warp) these input features to improve spatial invariance before the data is fed to a classification model.
- of an authentic product comprising an authentic product specification comparable with the subject product specification
[0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed. To facilitate verification of authenticity of an article, an article is commonly incorporated with an authentication device which includes an information bearing device such as a label, a tag or an imprint. The information bearing device comprises a data-embedded image pattern and the data-embedded image pattern is covertly encoded with a set of data so that the data is not perceivable by a reasonable person reading the data-embedded image pattern
Lau does not explicitly disclose:
- wherein the machine learning classifier is trained by a training dataset, wherein the training dataset comprises : i) extracted images, from a plurality of different camera types,
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics,
However, Pouyan discloses:
- wherein the machine learning classifier is trained by a training dataset, wherein the training dataset comprises : i) extracted images, from a plurality of different camera types,
[3.5.2, Page 115]:
The process for finding the network cameras depends on the types of the cameras. Some cameras have built-in web servers to distribute the data and each camera has a unique IP address.
[BRI: a network camera can be used for product authentication deployed with the right technology and integration that captures images of products at key points in the supply chain (e.g., on a production line, at a retail counter, or in a warehouse]
[4.1, Page 115]:
retrieve an image from every active camera in
[4.1, Page 116]:
the database.
a check program traverse the directory to remove the bad quality image based on their byte sizes. The final cleaned dataset contains over 10,000 images captured from network cameras.
[ 4.2, Page 116]:
CNN model plus a modified data augmentation in which each training batch includes a balanced number of classes
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics
[3.5.2, Page 115]:
Although many network cameras provide data to the public, finding them is not always easy because of the wide variety of brands and models. The data from many network cam eras are aggregated on web servers, and there are many different ways of organizing the data streams. To use the data from the network cameras, this system has a database of publicly available network cameras. The process for finding the network cameras depends on the types of the cameras.
[BRI: network camera refers to a surveillance network that contains multiple types of cameras, each designed for different environments, coverage needs, and image capture characteristics. In such a setup, you can indeed have three different camera types with distinct image capture characteristics] —
[1, Page 112]:
Millions of network cameras (a type of surveillance cameras) have been deployed in city streets, tourist attractions, and many other locations.
[1, Page 112]:
Figure 1 (a)-(d) show example images from network cameras, including roads, buildings, vehicles, and people. Figure 1 (e)-(i) show example images from several popular datasets. In contrast to the most commonly used datasets for classification, localization, ob ject detection, and segmentation, network cameras contain many objects that occupy small portions of an image and are not necessarily at the center. This paper introduces the first version of the network camera image dataset
PNG
media_image1.png
317
537
media_image1.png
Greyscale
[3.5.2, Page 115]:
The process for finding the network cameras depends on the types of the cameras. Some cameras have built-in web servers to distribute the data and each camera has a unique IP address. The data can be accessed by connecting directly to this address and using the HTTP GET requests. Different brands require different GET commands. For example, Axis cameras use /mjpg/video.mjpg but Foscam uses /video.cgi.
[BRI: a network camera can be used for product authentication deployed with the right technology and integration that captures images of products at key points in the supply chain (e.g., on a production line, at a retail counter, or in a warehouse]
[Abstract, Page 112]:
The proposed model can discover the semantic concepts from the data with a skewed distribution using a dynamic sampling technique. The paper also presents a system that can retrieve real-time visual data from heterogeneous cameras, and the run-time
environment allows the analysis programs to process the data from thousands of cameras simultaneously. The evaluation results in comparison with several state-of-the-art methods demonstrate the ability and effectiveness of the proposed model on visual data captured by public network cameras
[3.5.2, Page 115]:
Discovering Heterogeneous Network Cameras Although many network cameras provide data to the public, finding them is not always easy because of the wide variety of brands and models. The data from many network cam eras are aggregated on web servers, and there are many different ways of organizing the data streams.
[4.2, Page 116]:
The F1-score (Avg. F1) is adopted as the main evaluation metric since it is the most valuable comparison metric for imbalanced data
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, and Pouyan.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
One of ordinary skill would have motivation to combine Lau, and Pouyan that can provide improvement to the performance (classification) of the minority classes and maintain performance of the majority classes (Pouyan [1, Page 113]).
Lau and Pouyan do not explicitly disclose:
- A tangible, non-transitory computer-readable medium storing instructions storing instructions for imaging and classifying whether one or more physical and subject consumer goods are authentic or non-authentic, that when executed by one or more processors cause the one or more processors to:
- wherein the training dataset further comprises further extracted images, from the plurality of different camera types, of a non-authentic product comprising a non-authentic product specification
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types,
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
However, Krish discloses:
- A tangible, non-transitory computer-readable medium storing instructions storing instructions for imaging and classifying whether one or more physical and subject consumer goods are authentic or non-authentic, that when executed by one or more processors cause the one or more processors to:
[0043]:
As illustrated, embodiments of the enrollment/authentication system 110 and the POA system 210 can each include a respective instance of an image acquisition system 120, an image stack generator 130, a latent structural analyzer 140, and various data stores. The respective instances can be implemented identically or differently. For example, respective implementations of the image acquisition system 120 can specifically be adapted to the types of image acquisition components available to the respective system (e.g., each system can have a different type of camera, different optics capabilities, different resolutions, different lighting components and/or controls, etc.)
- wherein the training dataset further comprises further extracted images, from the plurality of different camera types, of a non-authentic product comprising a non-authentic product specification
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1 in more detail, in accordance with embodiments of the invention. However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras, video recorders, audio/video player, radio, GPS devices, wearable devices, Internet-of-things devices, augmented reality devices, virtual reality devices, automated teller machine devices, electronic kiosk devices, or any combination of the aforementioned.
[BRI: a mobile phone, wearable devices, AR include camera, a Camera itself can be a IoT device. Thus, all these can represent different types of cameras]
[0044]:
The user input devices 440, which allow the computing device system 400 to receive data from a user such as the user 110, may include any of a number of devices allowing the computing device system 400 to receive data from the user 110
[0044]:
The user interface may also include a camera 480, such as a digital camera.
[BRI: receiving the data from the camera represents image capture]
[0052]:
In an exemplary embodiment, the one or more advanced machine learning models may determine the authenticity of data by verifying that the data that is provided by a user for predicting the outcome is associated with a distribution that is used in training the one or more advanced models and may produce an outcome
[0059]:
As illustrated in block 610, the system executes instructions in the security module to (1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not. In one embodiment where the one or machine learning models identify that the data is not authentic, the one or more machine learning models do not produce an outcome. In another embodiment where the one or more machine learning models identify that the data is authentic, the one or more machine learning models produce an outcome.
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1 in more detail, in accordance with embodiments of the invention. However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras, video recorders, audio/video player, radio, GPS devices, wearable devices, Internet-of-things devices, augmented reality devices, virtual reality devices, automated teller machine devices, electronic kiosk devices, or any combination of the aforementioned.
[0060]:
As illustrated in block 615, the system executes instructions in the equality module to (1) provide disparity metrics to the one or more machine learning models, (2) provide bias tolerance levels to the one or more machine learning models, and (3) provide debiasing capabilities to the one or more machine learning models.
[0059]:
(1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
{BRI: by combining GAN-based data augmentation with centroid balancing, represents a robust training across a plurality of camera types.
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
[Abstract]:
Embodiments of the present invention provide a system for providing a centralized advanced security provisioning platform to create reliable machine learning models and also to enhance the existing machine learning models. The system is configured for executing instructions in the privacy module to monitor and control data privacy and data usage, executing instructions in the security module to preserve the authenticity of data that is used by the machine learning models to predict an outcome, executing instructions in the equality module to detect and prevent biasing of the machine learning models,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1
[0040]:
However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras
[BRI: the broader plurality of “cameras” represents types of cameras that may include but not limited to: DSLR, Mirrorless, Point and Shoot, Smartphone, Action, Webcams, etc]
[0052]:
machine learning models to identify whether the data that is being used by the one or more advanced machine learning models is authentic or not.
[0053]:
one or more advanced machine learning models will allow the one or more advanced machine learning models to identify whether the training data is genuine or not. The one or more advanced machine learning models may identify that the training data is genuine or not by calculating and comparing a centroid of the training data provided by the data scientists with a centroid of the raw training data
[0053]:
The method implemented by the one or more advanced machine learning models to detect whether the training data is genuine or not is centroid balancing as shown in layer 520 of FIG. 5.
[0059]:
(1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
[0060]:
(3) provide debiasing capabilities to the one or more machine learning models.
[0024]:
in a variety of fields and there are many applications that are closely related to our daily life, such as making significant decisions in application area based on predictions or classifications, in which a Machine Learning (ML) model could be relevant. Hence, if a ML model causes mispredictions or misclassifications due to malicious external influences, it can cause catastrophic complications.
[BRI: Perhaps known to the POSITA that misclassifications or mispredictions caused by malicious external influences can absolutely apply to product authenticity. In fact, such attacks are a growing concern in supply chain and verification systems. If attackers can manipulate the training data (data poisoning) or inject adversarial inputs (adversarial attacks), the model’s decision-making can be subtly altered. Attackers inject counterfeit product images or labels into training datasets so the model learns to misclassify genuine items as fake]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Shigeta and Krish.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Shigeta n teaches using at least three different camera types for capturing the image and manufacturing line code.
Krish teaches balanced training set across the camera types.
One of ordinary skill would have motivation to combine Lau, Shigeta and Krish that can provide enriched performance using machine learning in applications area (Krish[0024]) that includes authenticity [0052]).
Claims 12, and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over
Tak Wai Lau et. al (hereinafter Lau) US 2020/0410510 A1,
in view of Yasushi Shigeta et.al (hereinafter Shigeta) US 2019/0188958 A1,
in view of Madhusudan Krishnamoorthy et.al (hereinafter Krish) US 2020/0218825 A1.
In regard to claim 12: (Currently Amended )
Lau discloses:
- A machine learning based imaging method for imaging and classifying whether one or more physical and subject consumer goods are authentic or non-authentic, the machine learning based imaging method comprising:
[Abstract]:
An authentication apparatus and a method to devise an authentication tool is provided for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of an information bearing device on the article. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device using a trained neural network,
a) obtaining an image of a subject consumer good comprising a subject product specification
[0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed,
in [0070]:
A direct image of a source means the image is obtained directly from the source without intervening copying, that is, the image is not captured from an image of the source. An example source may be an information bearing device which is covertly coded with a security data designed to function as an authentic authentication device.
(BRI: The product specification is the source information on the information bearing device which is overtly coded)
- b) inputting the obtained image into a model, wherein the model is configured to classify the obtained image as authentic or non- authentic,
[0102]:
During the forward pass on progressing from an earlier layer to a next layer, each filter is convolved across the spatial dimensions of the input volume,
[0102]:
The entries of the filter collectively define a weight matrix and the weight matrix was learned by the CNN during deep learning training of the CNN”,
in [0187]:
The training images include images of authentic and non-authentic information bearing devices,
- wherein the model is constructed by a machine learning classifier,
[0075]:
An example CNN 30 comprises an input layer 300, an output layer 399, and a plurality of convolutional layers 301-30n interconnecting the input layer 300 and the output layer 399. CNN is a class of deep, feed-forward, artificial neural networks in machine learning,
- wherein the machine learning classifier is trained by a training dataset,
[0007]:
an authentication tool obtained by training a neural network using a large number of training images, say, several thousand training images, having controlled image imperfections substantially increases recognition rate and reliability of determination.
[0006]:
The training images comprises imperfect images having varying degrees of image imperfections. The image imperfections are controlled imperfections which are intentionally introduced into the imperfect images under controlled conditions in the course of image capture processes or due to different image capture conditions. The image imperfections are controlled within acceptable ranges which would facilitate meaningful training and determination of authenticity.
[0175]:
In an example, CNN is trained to learn to distinguish between an image of an authentic data-embedded image pattern and an image of a non-authentic data-embedded image pattern so that the trained CNN can provide classification information on the likelihood of whether an input image is an image of an authentic data-embedded image pattern or an image of a non-authentic data-embedded image pattern.
- c) outputting a classification output from the model indicating a likelihood that the image of the subject consumer good is authentic or non-authentic
[0079]:
The FCN will perform classification operations on the processed data of the CNN, for example, to determine whether, or how likely, the processed data of a target image CNN corresponds to an authentic authentication device or a non-authentic authentication device.
- and wherein the training dataset further comprises the extracted images of the authentic product augmented with geometric distortion so that the extracted images of the authentic product have a different shape.
In [0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed. To facilitate verification of authenticity of an article, an article is commonly incorporated with an authentication device which includes an information bearing device such as a label, a tag or an imprint. The information bearing device comprises a data-embedded image pattern and the data-embedded image pattern is covertly encoded with a set of data so that the data is not perceivable by a reasonable person reading the data-embedded image pattern. In example embodiments. Each data is a discrete data having characteristic two- or three-dimensional coordinate values in data domain and the coordinate values are transformed into spatial properties of image-defining elements which cooperate to define the entirety of the data-embedded image pattern. The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
(BRI: transforming data into spatial properties of image-defining elements (such as pixel positions, shapes, or sizes) and spreading coordinate values throughout those elements generally represents geometrical distortion)
In [0016] :
In some embodiments, the set of data embedded in the data-embedded image pattern comprises a plurality of discrete frequency data, and the discrete frequency data are transformed into spatially distributed pattern defining elements which are spread in the data-embedded image pattern and which are non-human readable or non-human perceivable using naked eyes; and the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
In [0136]:
Each pixel has characteristic physical properties including size, shape, color, brightness, etc., and the entirety of pixels collectively define a data-embedded image pattern.
(BRI: the process of spatially distributing a pattern that are non-human readable represents a sophisticated steganographic feature)
- and wherein the training dataset is spatially manipulated by a Spatial Transformer Network before training the machine learning classifier.
In [0006]:
The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
In [0016]:
the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
In [0177]:
On defining the CNN structure, the input layer is set to have a single channel since the example information bearing device 60 has a data-embedded image pattern which is defined by pattern defining elements in gray-scale coding.
(BRI: this process that combine frequency domain data with spatial manipulation, such as those used in Spectral-Spatial-Frequency Transformer Networks or Fourier-based data augmentation/feature extraction. Specifically, discrete frequency data (e.g., Fourier or Discrete Cosine Transform coefficients) can be transformed back into spatially distributed patterns or maps, which are then used to augment or inform training datasets. When these generated patterns are treated as input images, a Spatial Transformer Network (STN) can be employed to actively manipulate (e.g., rotate, scale, warp) these input features to improve spatial invariance before the data is fed to a classification model.
Lau does not explicitly disclose:
- wherein the non-authentic product specification is different from the Manufacturing Line Variable Printing Code;
- and (iii) an associated class definition based on the Manufacturing Line Variable Printing Code;
- wherein the non-authentic product specification is different from the Manufacturing Line Variable Printing Code;
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics,
However, Shigeta discloses:
- wherein the non-authentic product specification is different from the Manufacturing Line Variable Printing Code;
[0193]:
In addition, in the gaming currency 120, face codes using the UV ink or ink (carbon black ink) absorbing an infrared rays are arranged on the surface of the white layer 122 (see FIG. 9B). This face codes represent the authenticity of the gaming currency 120,
[0218]:
In addition, in a case where gaming currency 120 having side IDs is to be manufactured, side IDs are printed on the side faces of the white layers 122 or the thin-color layers through inkjet printing. Furthermore, in a case where gaming currency 120 including an RFID is to be manufactured, an RFID is interposed between the layers of the stacked structure body, and the layers having the RFID interposed therebetween are heated and welded when the layers are thermos-compressed, whereby the periphery of the RFID is tightly fixed by the plastic layers.
- and (iii) an associated class definition based on the Manufacturing Line Variable Printing Code;
[0282]:
On side faces of the gaming currency 120 manufactured in this way, side IDs 126 used for identifying each individual are printed. At this time, the side IDs 126 may be printed through inkjet printing.
[0249]:
in a case where ink is infrared reactive ink, and an ultraviolet camera in a case where ink is UV ink (UV radiator and a visible-light camera)) corresponding to the ink used for printing the side ID 126 or includes a camera capable of performing switching among a plurality of functions of cameras (a visible-light camera, an infrared ray camera, and an ultraviolet ray camera (an UV radiator and a visible-light camera or the like)) corresponding to the ink.
[0301]:
In a target recognition process, by performing an edge extracting process and the like for a recognition target image, candidate areas are extracted based on local features, and, after feature vectors are extracted by inputting the candidate areas to the convolutional neural network, classification is performed, and a candidate area having a classified highest certainty factor is acquired as a result of the recognition. Here, the certainty factor is an amount representing a relative degree of highness of the similarity of a subject of an image learned together with an image area and a label relative to the similarity of the other classes.
[BRI: For variable printing in a manufacturing setting, printing on packaging, labels, or other products using automated or semi-automated printing machines]
- wherein the non-authentic product specification is different from the Manufacturing Line Variable Printing Code;
[0261]:
The inspection device 300 includes: a face code reading device 306 that reads a face code M formed using UV emission ink or ink absorbing infrared rays disposed on the surface of the gaming currency 120;
[0252]:
Furthermore, in the gaming currency 120, a face code using UV ink or ink (carbon black ink) absorbing infrared rays is arranged on the surface (X) of the white layer 122 (see FIG. 9B). This face code represents authenticity of gaming currency 120, and, when ultraviolet rays (or infrared rays) hit the face code, the face code becomes visible to the eyes and represents authenticity based on a combination of the forms and a number.
[0267]:
the control device 207 is configured to inspect a relation among the information of the side ID 126 acquired by the ID reading device 307, the information acquired from the face code reading device 306, and the information acquired from the RFID reading device 308. There is a correct database (not illustrated in the drawing), and the relation among the information is inspected by comparing a reading result with the database. This inspection is used for preventing the occurrence of a defect by detecting a print error in the face code M and the side ID 126 or for discrimination with a fake
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics,
[0296]:
The side IDs 126 may be printed using two or more types of ink among ink (visible ink) that is visible for visible light, infrared-ray absorption ink, and UV emission ink. In such a case, the ID reading device 53 includes a plurality of types of cameras (a plurality of types of a visible-light camera, an infrared-ray camera, and an ultraviolet camera (a UV radiator and a visible-light camera, and the like) corresponding to ink used for printing the side IDs 126.
[0321]:
In the inspection device described above, the side ID may be printed using at least one or a combination of ink that is visible for visible light and ink absorbing infrared rays, and the ID reading device may include one or a plurality of visible-light cameras or infrared-ray cameras corresponding to the ink used for printing the side ID or a camera capable of performing switching among a plurality of functions of a visible-light camera and an infrared-ray camera corresponding to the ink.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, and Shigeta.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Shigeta teaches using at least three different camera types for capturing the image and manufacturing line code.
One of ordinary skill would have motivation to combine Lau, and Shigeta that provides an improved accuracy of the intelligent-type control device (Shigeta [0124]).
Lau and Shigeta do not explicitly disclose:
- wherein the machine learning classifier is trained by a training dataset, wherein the training dataset comprises: (i) extracted images, from a plurality of different camera types,
- wherein the training dataset further comprises further extracted images, from the plurality of different camera types, of a non-authentic product comprising a non-authentic product specification
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types,
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
However, Krish discloses:
- wherein the machine learning classifier is trained by a training dataset, wherein the training dataset comprises: (i) extracted images, from a plurality of different camera types,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1 in more detail, in accordance with embodiments of the invention. However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras, video recorders, audio/video player, radio, GPS devices, wearable devices, Internet-of-things devices, augmented reality devices, virtual reality devices, automated teller machine devices, electronic kiosk devices, or any combination of the aforementioned.
[BRI: a mobile phone, wearable devices, AR include camera, a Camera itself can be a IoT device. Thus, all these can represent different types of cameras]
[0044]:
The user input devices 440, which allow the computing device system 400 to receive data from a user such as the user 110, may include any of a number of devices allowing the computing device system 400 to receive data from the user 110
[0044]:
The user interface may also include a camera 480, such as a digital camera.
[BRI: receiving the data from the camera represents image capture]
[0059]:
Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
- wherein the training dataset further comprises further extracted images, from the plurality of different camera types, of a non-authentic product comprising a non-authentic product specification
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1 in more detail, in accordance with embodiments of the invention. However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras, video recorders, audio/video player, radio, GPS devices, wearable devices, Internet-of-things devices, augmented reality devices, virtual reality devices, automated teller machine devices, electronic kiosk devices, or any combination of the aforementioned.
[BRI: a mobile phone, wearable devices, AR include camera, a Camera itself can be a IoT device. Thus, all these can represent different types of cameras]
[0044]:
The user input devices 440, which allow the computing device system 400 to receive data from a user such as the user 110, may include any of a number of devices allowing the computing device system 400 to receive data from the user 110
[0044]:
The user interface may also include a camera 480, such as a digital camera.
[BRI: receiving the data from the camera represents image capture]
[0052]:
In an exemplary embodiment, the one or more advanced machine learning models may determine the authenticity of data by verifying that the data that is provided by a user for predicting the outcome is associated with a distribution that is used in training the one or more advanced models and may produce an outcome
[0059]:
As illustrated in block 610, the system executes instructions in the security module to (1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not. In one embodiment where the one or machine learning models identify that the data is not authentic, the one or more machine learning models do not produce an outcome. In another embodiment where the one or more machine learning models identify that the data is authentic, the one or more machine learning models produce an outcome.
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1 in more detail, in accordance with embodiments of the invention. However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras, video recorders, audio/video player, radio, GPS devices, wearable devices, Internet-of-things devices, augmented reality devices, virtual reality devices, automated teller machine devices, electronic kiosk devices, or any combination of the aforementioned.
[0060]:
As illustrated in block 615, the system executes instructions in the equality module to (1) provide disparity metrics to the one or more machine learning models, (2) provide bias tolerance levels to the one or more machine learning models, and (3) provide debiasing capabilities to the one or more machine learning models.
[0059]:
(1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
{BRI: by combining GAN-based data augmentation with centroid balancing, represents a robust training across a plurality of camera types.
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
[Abstract]:
Embodiments of the present invention provide a system for providing a centralized advanced security provisioning platform to create reliable machine learning models and also to enhance the existing machine learning models. The system is configured for executing instructions in the privacy module to monitor and control data privacy and data usage, executing instructions in the security module to preserve the authenticity of data that is used by the machine learning models to predict an outcome, executing instructions in the equality module to detect and prevent biasing of the machine learning models,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1
[0040]:
However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras
[BRI: the broader plurality of “cameras” represents types of cameras that may include but not limited to: DSLR, Mirrorless, Point and Shoot, Smartphone, Action, Webcams, etc]
[0052]:
machine learning models to identify whether the data that is being used by the one or more advanced machine learning models is authentic or not.
[0053]:
one or more advanced machine learning models will allow the one or more advanced machine learning models to identify whether the training data is genuine or not. The one or more advanced machine learning models may identify that the training data is genuine or not by calculating and comparing a centroid of the training data provided by the data scientists with a centroid of the raw training data
[0053]:
The method implemented by the one or more advanced machine learning models to detect whether the training data is genuine or not is centroid balancing as shown in layer 520 of FIG. 5.
[0059]:
(1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
[0060]:
(3) provide debiasing capabilities to the one or more machine learning models.
[0024]:
in a variety of fields and there are many applications that are closely related to our daily life, such as making significant decisions in application area based on predictions or classifications, in which a Machine Learning (ML) model could be relevant. Hence, if a ML model causes mispredictions or misclassifications due to malicious external influences, it can cause catastrophic complications.
[BRI: Perhaps known to the POSITA that misclassifications or mispredictions caused by malicious external influences can absolutely apply to product authenticity. In fact, such attacks are a growing concern in supply chain and verification systems. If attackers can manipulate the training data (data poisoning) or inject adversarial inputs (adversarial attacks), the model’s decision-making can be subtly altered. Attackers inject counterfeit product images or labels into training datasets so the model learns to misclassify genuine items as fake]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Shigeta and Krish.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Shigeta n teaches using at least three different camera types for capturing the image and manufacturing line code.
Krish teaches balanced training set across the camera types.
One of ordinary skill would have motivation to combine Lau, Shigeta and Krish that can provide enriched performance using machine learning in applications area (Krish[0024]) that includes authenticity [0052]).
In regard to claim 14: (Previously Presented)
Lau discloses:
- augmenting the extracted images of the authentic product with color distortion
[0136]:
Modern authentic information bearing devices contain data-embedded image patterns which are digitally formed and consist of pixels. Each pixel has characteristic physical properties including size, shape, color, brightness, etc
[0136]:
Some of the characteristic physical properties suffer degradation or loss of fidelity during image capture and/or reproduction,
[0138]:
When the data-embedded image pattern of the authentic information bearing device is captured by an image capture apparatus, the gray levels of the pixels forming the captured image may be changed. For example, the gray levels may be shifted linearly, non-linearly, randomly or may have an entirely different gray-scale distribution of pixels compared to those of the data-embedded image pattern. The change may be due to internal setting of the image capture apparatus (for example, exposure setting), calibration of the image capture apparatus, ambient illumination, sensitivity and/or linearity of the image sensor of the capture apparatus, angle of image capture, and/or other parameters.
In regard to claim 15: (Original)
Lau do not explicitly disclose:
- wherein the Manufacturing Line Variable Printing Code comprises one or more of: one or more alphanumeric characters, one or more non-alphanumeric characters, one or more non-alphanumeric characters comprising a pattern box, or one or more non-alphanumeric characters comprising a dotted column.
However, Shigeta discloses:
- wherein the Manufacturing Line Variable Printing Code comprises one or more of: one or more alphanumeric characters, one or more non-alphanumeric characters, one or more non-alphanumeric characters comprising a pattern box, or one or more non-alphanumeric characters comprising a dotted column.
[0249]:
The gaming currency 120 forms a stripe pattern on the side face in the stacking direction by employing a multi-layer structure
[0248]:
Particularly, in this embodiment, the side ID 126 is configured to be attached as presence/absence of marks C of a plurality of rows and a plurality of columns. In the marks C of the plurality of rows and the plurality of columns, as illustrated in FIG. 17, upper and lower marks C are paired to configure a code, and a code of ten digits is formed in the case illustrated in FIG. 17. A configuration in which upper and lower marks C are paired to configure a code (four types) is illustrated in FIG. 18. A letter “Y” disposed to the side of marks C is an identification mark used for identifying upper and lower sides of a mark. A code configured by marks C is configured to specify a predetermined combination of marks C.
[0250]:
In the case illustrated in FIG. 19, in gaming currency 120-1, side IDs 126 are printed using ink (ink absorbing infrared rays) that is not visible for visible light. In gaming currency 120-2, marks C of a plurality of rows and a plurality of columns are printed using ink that is seen dark for visible light.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, and Shigeta.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Shigeta teaches using at least three different camera types for capturing the image and manufacturing line code.
One of ordinary skill would have motivation to combine Lau, and Shigeta that provides an improved accuracy of the intelligent-type control device (Shigeta [0124]).
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over
Tak Wai Lau et. al (hereinafter Lau) US 2020/0410510 A1,
in view of Steven Perna et.al (hereinafter Perna) US 2015/0098630 A1,
in view of Madhusudan Krishnamoorthy et.al (hereinafter Krish) US 2020/0218825 A1.
In regards to claim 22: (Currently Amended)
Lau discloses:
- A machine learning based imaging system configured to image and classify whether one or more physical and subject consumer goods are authentic or non-authentic,
[0013]:
A self-learning system and methods for automatic document classification, authentication, and information extraction are described.
[0025]:
This invention uses unique methods to automatically train classes of documents, i.e., it allows the training subsystem to self-learn optimal parameters for classification and authentication. Thereby, it improves the accuracy and reliability, and shortens the training time.
- image and classify whether one or more physical and subject consumer goods are authentic or non-authentic, the machine learning based imaging system comprising
in [Abstract]:
“An authentication apparatus and a method to devise an authentication tool is provided for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of an information bearing device on the article. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device using a trained neural network;
(BRI: Information bearing device is a product)
- a) obtain an image of a subject consumer good comprising a subject product specification, the image captured by the mobile device;
in [0070] :
A direct image of a source means the image is obtained directly from the source without intervening copying, that is, the image is not captured from an image of the source. An example source may be an information bearing device which is covertly coded with a security data designed to function as an authentic authentication device.
- b) input the obtained image into a model, wherein the model is configured to classify the obtained image as authentic or non- authentic,
[0102] :
During the forward pass on progressing from an earlier layer to a next layer, each filter is convolved across the spatial dimensions of the input volume,
[0102]:
The entries of the filter collectively define a weight matrix and the weight matrix was learned by the CNN during deep learning training of the CNN,
in [0187]:
The training images include images of authentic and non-authentic information bearing devices,
- wherein the model is constructed by a machine learning classifier,
[0075] :
An example CNN 30 comprises an input layer 300, an output layer 399, and a plurality of convolutional layers 301-30n interconnecting the input layer 300 and the output layer 399. CNN is a class of deep, feed-forward, artificial neural networks in machine learning,
- wherein the machine learning classifier is trained by a training dataset,
[0183] :
The training images are selected according to some selection criteria such that the imperfection values are within the acceptable ranges,
in [0187]:
The training images include images of authentic and non-authentic information bearing devices.
[0072]:
Due to its unique properties, for example, a specific or one-to-one correspondence between a set of data and a set of spatial image pattern having spread or distributed pattern defining elements to represent the set of data,
in [0072]:
the information bearing device can be used as an authentication device, with the encoded coordinate data or encoded set of coordinate data,
[0073] :
An authentic authentication device herein is also referred to as an authentic information bearing device or a genuine information bearing device herein, while a non-authentic authentication device is also referred to as a non-authentic information bearing device or a non-genuine information bearing device where appropriate.
- the authentic product specification comprises at least one steganographic feature having a length greater than 0.01 mm;
[0108]:
the example information bearing device 60 is set to have an physical size of a one-cm square,
[0109]:
To print the information bearing device using a 1200 DPI printer on a 1 cm× 1 cm medium, the information bearing device 60 need to be resized and quantized. Specifically, the information bearing device is needed to resize to a width and height of 472 pixels in each orthogonal direction, since 472 pixels per cm is equivalent to 1200 DPI. Each pixel of the data-embedded image pattern is a real number and the resized information bearing device is quantized from real number to bi-level,
[0070] :
An example source may be an information bearing device which is covertly coded with a security :
data designed to function as an authentic authentication device,
[0070] :
The covertly coded data is typically not human readable or perceivable and the data coding may be by means of steganographic techniques such as transform domain coding techniques.
- (ii) an associated class definition based on the steganographic feature;
[0080]:
An example CNN of an example authentication apparatus comprises a plurality of convolution layers between the input layer and the output layer, as depicted in FIG. 3. The convolution layers of the CNN are serially connected to form an ensemble of serially connected convolution layers. Each convolution layer comprises a plurality of filters, and each filter is a convolution filter which is to operate with an input data file to generate an output data file. A plurality of output data files is generated as a result of convolution operations among the convolution filters and the input data files at the input of a convolution layer. Each output data file is referred to as a feature map in CNN terminology.
[0079]:
The fully connected network (“FCN”) is connected to output of the CNN, such that output of the CNN is fed as input to the FCN, as depicted in FIG. 2. The FCN will perform classification operations on the processed data of the CNN, for example, to determine whether, or how likely, the processed data of a target image CNN corresponds to an authentic authentication device or a non-authentic authentication device.
- c) output a classification output from the model indicating a likelihood that the image of the subject consumer good is authentic or non-authentic
[0079:
The FCN will perform classification operations on the processed data of the CNN, for example, to determine whether, or how likely, the processed data of a target image CNN corresponds to an authentic authentication device or a non-authentic authentication device.
- wherein the training dataset further comprises further extracted image, from the plurality of different camera types, of a non-authentic product comprising a non-authentic product specification, wherein the non-authentic product specification is different from the at least one steganographic feature,
[0073]:
An authentic authentication device herein is also referred to as an authentic information bearing device or a genuine information bearing device herein, while a non-authentic authentication device is also referred to as a non-authentic information bearing device or a non-genuine information bearing device where appropriate. The target image may be captured by the apparatus or received from an outside source.
[0132]:
An image of an authentic authentication device is a primary copy of an authentic information bearing device, while an image of a non-authentic authentication device may be a secondary copy of an authentic information bearing device or a copy of a fake information bearing device.
[0071]:
An example information bearing device herein comprises a data-encoded image pattern which is encoded with a set of discrete data. The set of data is human non-perceivable in its encoded state such that the data is not readily readable or readily decodable by a human reader looking at the data-encoded image pattern using naked eyes.
- and wherein the training dataset further comprises the extracted images of the authentic product augmented with geometric distortion so that the extracted images of the authentic product have a different shape.
[0006]:
An authentication tool, an authentication apparatus and a method to devise an authentication tool for facilitating determination of authenticity or genuineness of an article with reference to a captured image or a purported primary image of the article using a neural network is disclosed. To facilitate verification of authenticity of an article, an article is commonly incorporated with an authentication device which includes an information bearing device such as a label, a tag or an imprint. The information bearing device comprises a data-embedded image pattern and the data-embedded image pattern is covertly encoded with a set of data so that the data is not perceivable by a reasonable person reading the data-embedded image pattern. In example embodiments. Each data is a discrete data having characteristic two- or three-dimensional coordinate values in data domain and the coordinate values are transformed into spatial properties of image-defining elements which cooperate to define the entirety of the data-embedded image pattern. The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
[0016] :
In some embodiments, the set of data embedded in the data-embedded image pattern comprises a plurality of discrete frequency data, and the discrete frequency data are transformed into spatially distributed pattern defining elements which are spread in the data-embedded image pattern and which are non-human readable or non-human perceivable using naked eyes; and the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
[0136]:
Each pixel has characteristic physical properties including size, shape, color, brightness, etc., and the entirety of pixels collectively define a data-embedded image pattern.
- and wherein the training dataset is spatially manipulated by a Spatial Transformer Network before training the machine learning classifier.
[0006]:
The spatial properties include, for example, brightness or amplitude of an image-defining element at a specific set of coordinates on the data domain. The data may be covertly coded by a transformation function which operate to spread the coordinate values of a data into spatial properties spread throughout the image-defining elements. The set of data or each individual discrete data point has characteristic signal strengths. The authenticity or genuineness of an article is determined with reference to whether a captured image is a primary image of an authentic information bearing device.
[0016]:
the spatially distributed pattern defining elements and the discrete frequency data are correlated by Fourier transform.
[0177]:
On defining the CNN structure, the input layer is set to have a single channel since the example information bearing device 60 has a data-embedded image pattern which is defined by pattern defining elements in gray-scale coding.
Lau does not explicitly disclose:
- the machine learning based imaging system comprising: a server comprising a processor and a memory, the memory storing a model;
- and a software application (app) configured to execute on a mobile device comprising a mobile processor and a mobile memory, the software app communicatively coupled to the server via a computer network, wherein the server comprises computing instructions configured for execution on the processor, and that when executed by the processor causes the processor to:
However, Perna discloses:
- the machine learning based imaging system comprising: a server comprising a processor and a memory, the memory storing a model;
[0032]:
FIG. 1 depicts a block diagram of an iris processor 100 for biometric iris matching
[0032]:
The iris processor 100 comprises a pre-processor 102, a coding processor 104 and a matching processor 106. The iris processor 100 receives images as input, for example, input image 101 and outputs a matched iris 108 from a remote or local database.
[0032]:
the coding processor 104 and the matching processor 106 may execute on a single device, or on different devices, servers, cloud services or the like,
[0032]:
The iris processor 100 may be modular and each processor may be implemented, e.g., on a single device, multiple devices, in the cloud as a service.
[0037]:
the image captured by the camera to a server where the pre-processor 102 is executed for pre-processing
[0096]:
The mobile device may contain a camera and have the iris processor 1206 stored on memory as an application
[0069]:
An example of two such histograms is shown in FIG. 7. The histogram on the left corresponds to an impostor match and the one on the right to an authentic match.
[0069]:
Furthermore, give sufficient training sets of impostor and authentic histograms it may be beneficial to use statistical classification or machine learning techniques such as discriminant analysis, Support Vector Machines, Neural Networks, or Logistic Regression to construct an optimal decision procedure for some class of data.
[0037]:
The matched iris data 108 may be used in many instances, for example, to authorize financial transactions. The pre-processor 102 may be an application executing on a mobile device, such as a mobile phone, camera, tablet, or the like. The pre-processor 102 on the mobile device may capture an image of a user's eye using the camera of the device, perform the pre-processing steps on the mobile device, and then transmit a bundled and encrypted request to the coding processor 104, which may be accessed via a cloud service on a remote server
[BRI: Perhaps known to a POSITA, data access via a cloud service almost always requires that the data be stored on the remote server.
- and a software application (app) configured to execute on a mobile device comprising a mobile processor and a mobile memory, the software app communicatively coupled to the server via a computer network, wherein the server comprises computing instructions configured for execution on the processor, and that when executed by the processor causes the processor to:
[0037]:
The matched iris data 108 may be used in many instances, for example, to authorize financial transactions. The pre-processor 102 may be an application executing on a mobile device, such as a mobile phone, camera, tablet, or the like. The pre-processor 102 on the mobile device may capture an image of a user's eye using the camera of the device, perform the pre-processing steps on the mobile device, and then transmit a bundled and encrypted request to the coding processor 104, which may be accessed via a cloud service on a remote server
[0037]:
In other embodiments, the application on the mobile device may also comprise the coding processor 104 and the iris coding is performed on the mobile device. In some embodiments, the pre-processor 102 may be used in conjunction with an automated teller machine (ATM), where a user is authorized via their iris being scanned and processed by the pre-processor 102. The pre-processor 102 may then reside in the software of the ATM, or the ATM may supply the image captured by the camera to a server where the pre-processor 102 is executed for pre-processing.
[0038]:
The matching processor 106 may be hosted on a server of a financial institution, or be a remote third party service available to multiple financial institutions for authenticating the user based on their iris image.
[0038]:
the iris processor 100 may be used to authenticate a user in any context, such as signing in to a social network, a messaging service or the like.
0096]:
The computer system 1200 may be a mobile device such as a cellular phone or tablet device, for example. The mobile device may contain a camera and have the iris processor 1206 stored on memory as an application
- wherein the machine learning classifier is trained by a training dataset, wherein the training dataset comprises: (i) extracted images, from a plurality of different camera types,
of an authentic product comprising an authentic product specification comparable with the subject product specification,
[0069]:
An example of two such histograms is shown in FIG. 7. The histogram on the left corresponds to an impostor match and the one on the right to an authentic match.
[0069]:
there are many other measures of central concentration and dispersion that may be used to distinguish between authentic and impostor distributions
[0069]:
Furthermore, give sufficient training sets of impostor and authentic histograms it may be beneficial to use statistical classification or machine learning techniques such as discriminant analysis, Support Vector Machines, Neural Networks, or Logistic Regression to construct an optimal decision procedure for some class of data.
[0038]:
The matching processor 106 may be hosted on a server of a financial institution, or be a remote third party service available to multiple financial institutions for authenticating the user based on their iris image.
[0038]:
the iris processor 100 may be used to authenticate a user in any context, such as signing in to a social network, a messaging service or the like.
[0096:
The computer system 1200 may be a mobile device such as a cellular phone or tablet device, for example. The mobile device may contain a camera and have the iris processor 1206 stored on memory as an application. In some embodiments, the iris processor 1206 may be a part of the operating system 1220.
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics,
[0096]:
often mobile devices have camera processing modules and the iris processor 1206, or portions of the iris processor 1206, may reside on the camera processing module, where the imager in the camera is a CCD or CMOS imager. In some instances, the mobile device may be customized to include some sensors, the type of the camera imager, or the like.
[0099]:
use of a different type of imaging device, lighting arrangement, or optics can vary the dimensions of the capture zone 1420.
- wherein the plurality of different camera types comprises at least three different camera types having different image capture characteristics
[0101]:
the iris biometric recognition module 1414 obtains an image of the face and eyes of the human subject 1424 using an imaging device (e.g., one or more digital cameras).
[0110]:
In some embodiments, both the face imager 1648 and the iris imager 1644 utilize the same type of imager (e.g., a digital camera, such as the Omnivision model no. OV02643-A42A), equipped with different lenses. For example, the face imager 1648 may be equipped with a wide field of view lens such as the Senview model no. TN01920B and the iris imager 1644 may be equipped with a narrow field of view lens such as model no. JHV-8M-85 by JA HWA Electronics Co. In other embodiments, a single high resolution imager (e.g., a 16+ megapixel digital camera) may be used with a wide field of view lens (rather than a combination of two cameras with different lenses) to perform the functionality of the iris imager 1644 and the face imager 1648.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, and Perna.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Perna teaches server and mobile system at least three different camera types for capturing the image.
One of ordinary skill would have motivation to combine Lau, and Perna that can use different of imaging device (camera) to increase the size of the capture zone (Perna [0099]) that may provide the authentication of the user [0038]).
Lau and Perna do not explicitly disclose:
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types,
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
However, Krish discloses:
- and wherein the machine learning classifier is trained so that the model is robust to differences in image capture associated with the plurality of different camera types,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1 in more detail, in accordance with embodiments of the invention. However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras, video recorders, audio/video player, radio, GPS devices, wearable devices, Internet-of-things devices, augmented reality devices, virtual reality devices, automated teller machine devices, electronic kiosk devices, or any combination of the aforementioned.
[0060]]:
As illustrated in block 615, the system executes instructions in the equality module to (1) provide disparity metrics to the one or more machine learning models, (2) provide bias tolerance levels to the one or more machine learning models, and (3) provide debiasing capabilities to the one or more machine learning models.
[0059]:
(1) train the one or more machine learning models using Generative Adversarial Networks, and (2) provide centroid balancing capabilities to the one or more machine learning models. Training the one or more machine learning models using the Generative Adversarial Networks allows the one or more machine learning models to identify whether the data that is being used by the one or more machine learning models to predict an outcome is authentic or not.
{BRI: by combining GAN-based data augmentation with centroid balancing, represents a robust training across a plurality of camera types.
- and wherein the training data set further comprises a balanced set of extracted images comprising both authentic and non-authentic corresponding consumer goods, wherein the training dataset is balanced across authentic and non-authentic classes and each of the plurality of different camera types
[Abstract]:
Embodiments of the present invention provide a system for providing a centralized advanced security provisioning platform to create reliable machine learning models and also to enhance the existing machine learning models. The system is configured for executing instructions in the privacy module to monitor and control data privacy and data usage, executing instructions in the security module to preserve the authenticity of data that is used by the machine learning models to predict an outcome, executing instructions in the equality module to detect and prevent biasing of the machine learning models,
[0040]:
FIG. 4 provides a block diagram illustrating a computing device system 400 of FIG. 1
[0040]:
However, it should be understood that a mobile telephone is merely illustrative of one type of computing device system 400 that may benefit from, employ, or otherwise be involved with embodiments of the present invention and, therefore, should not be taken to limit the scope of embodiments of the present invention. Other types of computing devices may include portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, desktop computers, workstations, laptop computers, cameras
[BRI: the broader plurality of “cameras” represents types of cameras that may include but not limited to: DSLR, Mirrorless, Point and Shoot, Smartphone, Action, Webcams, etc]
[0052]:
machine learning models to identify whether the data that is being used by the one or more advanced machine learning models is authentic or not.
[0053]:
one or more advanced machine learning models will allow the one or more advanced machine learning models to identify whether the training data is genuine or not. The one or more advanced machine learning models may identify that the training data is genuine or not by calculating and comparing a centroid of the training data provided by the data scientists with a centroid of the raw training data
[0053]:
The method implemented by the one or more advanced machine learning models to detect whether the training data is genuine or not is centroid balancing as shown in layer 520 of FIG. 5.
[0060]:
(3) provide debiasing capabilities to the one or more machine learning models.
[0024]:
if a ML model causes mispredictions or misclassifications due to malicious external influences, it can cause catastrophic complications.
[BRI: Perhaps known to the POSITA that misclassifications or mispredictions caused by malicious external influences can absolutely apply to product authenticity. In fact, such attacks are a growing concern in supply chain and verification systems. If attackers can manipulate the training data (data poisoning) or inject adversarial inputs (adversarial attacks), the model’s decision-making can be subtly altered. Attackers inject counterfeit product images or labels into training datasets so the model learns to misclassify genuine items as fake]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Perna and Krish.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Perna teaches server and mobile system at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
One of ordinary skill would have motivation to combine Lau, Perna and Krish that can provide enriched performance using machine learning in applications area (Krish[0024])
Claims 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over
Tak Wai Lau et. al (hereinafter Lau) US 2020/0410510 A1,
in view of Yasushi Shigeta et.al (hereinafter Shigeta) US 2019/0188958 A1,
in view of Madhusudan Krishnamoorthy et.al (hereinafter Krish) US 2020/0218825 A1.
further in view of Simske et.al (hereinafter Simske) US 2011/0280480 A1.
In regard to claim 16: (Previously Presented)
Lau, Pouyan and Krish do not explicitly disclose:
- Manufacturing Line Variable Printing Code is printed or affixed to the subject consumer good by one or more of: a continuous ink-jet printer, an embossing, a laser etching, thermal transferring, or hot waxing
However, Simske discloses:
- Manufacturing Line Variable Printing Code is printed or affixed to the subject consumer good by one or more of: a continuous ink-jet printer, an embossing, a laser etching, thermal transferring, or hot waxing
[0013]:
As non-limiting examples, the indicia 24 may be formed of inkjet ink, laserjet ink, spectrally opaque ink, spectrally transparent ink, ultraviolet ink, infrared ink, thermochromatic ink, electrochromatic ink, electroluminescent ink, conductive ink, magnetic ink, color-shifting ink, quantum dot ink, phosphorescent ink, a guilloche, a planchette, holographs, security threads, watermarks, other security deterrents, anti-tamper deterrents, and combinations thereof.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Simske.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Simske teaches printing code.
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Simske to determine an authencity of a product using steganographic feature in the image for variety of applications to enhance anti-counterfeit efforts (Simske [0045]).
In regard to claim 17: (Previously Presented)
Lau, Pouyan and Krish do not explicitly disclose:
- training dataset comprises annotations that annotate the Manufacturing Line Variable Printing Code
However, Simske discloses:
- training dataset comprises annotations that annotate the Manufacturing Line Variable Printing Code
[0006]:
the detection of the steganographic marks may be used by brand protection investigators to process many images simultaneously and discover counterfeit images in large data sets; the detection of variable data printing regions may be used for proofing and/or inspecting in print authentication; and the detection of low quality marks may be used for proofing, print defect detection, and auditing.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Simske.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Simske teaches printing code.
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Simske to determine an authencity of a product using steganographic feature in the image for variety of applications to enhance anti-counterfeit efforts (Simske [0045]).
Claims 6-7 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over
Tak Wai Lau et. al (hereinafter Lau) US 2020/0410510 A1,
in view of Samira Pouyanfar et.al (hereinafter Pouyan), Dynamic Sampling in Convolutional Neural Networks for Imbalanced Data Classification, 2018 IEEE Conference on Multimedia Information Processing and Retrieval.
in view of Madhusudan Krishnamoorthy et.al (hereinafter Krish) US 2020/0218825 A1.
further in view of Simske et.al (hereinafter Simske) US 2011/0280480 A1.
In regard to claim 6: (Original)
Lau, Pouyan and Krish do not explicitly disclose:
- the at least one steganographic feature is selected from one or more of: an isolated font style for a letter, an isolated font style for a number; an isolated location change of a text location, an isolated location change of a letter location, an isolated location change of a punctuation location
However, Simske discloses:
- the at least one steganographic feature is selected from one or more of: an isolated font style for a letter, an isolated font style for a number; an isolated location change of a text location, an isolated location change of a letter location, an isolated location change of a punctuation location
[0012]:
In another non-limiting example, the indicia 24 are security deterrents SD (some of which may be steganographic, i.e., capable of having information hidden therein) selected from color lines, fingerprints, color text, copy detection patterns (CDP), color tiles, letter sequences, number sequences, graphic sequences, target patterns, bar codes, and the like, and combinations thereof.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Simske.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Simske teaches steganographic features .
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Simske to determine an authencity of a product using steganographic feature in the image for variety of applications to enhance anti-counterfeit efforts (Simske [0045]).
In regard to claim 7: (Original)
Lau, Pouyan and Krish do not explicitly disclose:
- the authentic product specification is selected from one or more of: a production code, a batch code, a brand name, a product line, a label, artwork, an ingredient list, or usage instructions
However, Simske discloses:
- the authentic product specification is selected from one or more of: a production code, a batch code, a brand name, a product line, a label, artwork, an ingredient list, or usage instructions
[0012] :
The indicia 24 printed on the object 22 may include, but are not limited to graphical indicia, alphanumeric indicia, or combinations thereof. In one non-limiting example, the indicia 24 are text T or images I which include brand information, product information, manufacturer or distributor information, and/or any other desirable textual and/or graphical information.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Simske.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Simske teaches steganographic features .
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Simske to determine an authencity of a product using steganographic feature in the image for variety of applications to enhance anti-counterfeit efforts (Simske [0045]).
In regard to claim 18: (Original)
Lau, Pouyan and Krish do not explicitly disclose:
- the training dataset comprises annotations annotating the at least one steganographic feature
However, Simske discloses:
- the training dataset comprises annotations annotating the at least one steganographic feature
[0006]:
the detection of the steganographic marks may be used by brand protection investigators to process many images simultaneously and discover counterfeit images in large data sets; the detection of variable data printing regions may be used for proofing and/or inspecting in print authentication; and the detection of low quality marks may be used for proofing, print defect detection, and auditing.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Simske.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Simske teaches steganographic features .
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Simske to determine an authencity of a product using steganographic feature in the image for variety of applications to enhance anti-counterfeit efforts (Simske [0045]).
In regard to claim 19: (Previously Presented)
Lau, Pouyan and Krish do not explicitly disclose:
- the at least one steganographic feature is generated by computing instructions configured for execution on a processor, that when executed caused the processor to automatically generate the steganographic feature based on one or more steganographic feature types
However, Simske discloses:
- the at least one steganographic feature is generated by computing instructions configured for execution on a processor, that when executed caused the processor to automatically generate the steganographic feature based on one or more steganographic feature types
[0007]:
These components of the system 10 are part of a computer or enterprise computing system 20, which includes programs or software configured to segment an image, store and retrieve previously saved templates and/or zoning output specifications, store and retrieve previously stored region of interest information and strategies, and identify one or more regions of interest of the image”,
[0029] :
As still another non-limiting example, if the end-application is determining steganographic content areas, then the strategy may involve looking for small regions, which may be noted in the registry 16 as being good candidates for the selected region of interest for such applications.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Simske.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Simske teaches steganographic features .
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Simske to determine an authencity of a product using steganographic feature in the image for variety of applications to enhance anti-counterfeit efforts (Simske [0045]).
In regard to claim 20: (Original)
Lau, Pouyan and Krish do not explicitly disclose:
- the at least one steganographic feature is affixed on the subject consumer good during or after manufacture of the subject consumer good
However, Simske discloses:
- the at least one steganographic feature is affixed on the subject consumer good during or after manufacture of the subject consumer good
[0026]:
The strategies may be developed and saved after one image has been deployed and analyzed, inspected, authenticated, or the like, and may be changed and/or refined over time. It is to be understood that any type of machine learning may be employed here,
[0012]:
The indicia 24 printed on the object 22 may include, but are not limited to graphical indicia, alphanumeric indicia, or combinations thereof. In one non-limiting example, the indicia 24 are text T or images I which include brand information, product information, manufacturer or distributor information, and/or any other desirable textual and/or graphical information. In another non-limiting example, the indicia 24 are security deterrents SD (some of which may be steganographic, i.e., capable of having information hidden therein) selected from color lines, fingerprints, color text, copy detection patterns (CDP), color tiles, letter sequences, number sequences, graphic sequences, target patterns, bar codes, and the like, and combinations thereof.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Simske.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Simske teaches steganographic features .
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Simske to determine an authencity of a product using steganographic feature in the image for variety of applications to enhance anti-counterfeit efforts (Simske [0045]).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over
Tak Wai Lau et. al (hereinafter Lau) US 2020/0410510 A1,
in view of Samira Pouyanfar et.al (hereinafter Pouyan), Dynamic Sampling in Convolutional Neural Networks for Imbalanced Data Classification, 2018 IEEE Conference on Multimedia Information Processing and Retrieval.
in view of Madhusudan Krishnamoorthy et.al (hereinafter Krish) US 2020/0218825 A1.
further in view of Ronald Bruce Blair et.al (hereinafter Blair) US 2014/0037196 A1.
In regards to claim 23: (Previously Presented)
Lau, Pouyan and Krish do not explicitly disclose:
- wherein the training data set comprises [[the]] approximately equal number of extracted images from both authentic and non- authentic corresponding consumer goods such that a number of the extracted images from the authentic consumer goods is within 5% or less of a number the extracted images from the non- authentic consumer goods.
However, Blair discloses:
- wherein the training data set comprises the approximately equal number of extracted images from both authentic and non- authentic corresponding consumer goods such that a number of the extracted images from the authentic consumer goods is within 5% or less of a number the extracted images from the non- authentic consumer goods.
[0034]:
Any suitable image sensor 104 capable of capturing any suitable image (frame, line, or otherwise) of a document may be employed
[0023]:
The training intensity values 128, 130 may be obtained from at least one training document 132 that is used as a benchmark or model to determine whether the document 116 is authentic.
[0023]:
A training module 133 may capture one or more images 134, 136 of a training region 138 of the training document 132 in conjunction with the image capturing module 106, the light source 102, and the image sensor 104. In another embodiment, the training module 133 may be separate from the document authentication application 108 and be implemented by a different device, authority, or entity than that used to process and authorize the document 116. The training images 134, 136 may undergo processing to determine the training intensity values 123, 130 for each of the training images 130, 136, respectively. To provide comparisons between wavelength-dependent intensities of the training document 132 and the document 116, the images 110, 112 may be captured at the same or similar wavelengths as the training images 134, 136 of the training document 132.
[0023]:
Also, in one embodiment, training data may be collected from a plurality of training documents, whereby an average or acceptable range may be determined for comparison with the images 110, 112.
[0024]:
The training intensity value 128 is indicative of an intensity of the image 134 and the training intensity value 130 is indicative of an intensity of the image 136. In one embodiment, the training intensity values 128, 130 may include a mean training intensity and a standard deviation of the mean training intensity for the images 134 and 136, respectively. However, other values, such as the median, maximum, minimum, average, etc., indicative of or associated with the intensity of the pixels may be used for the training intensity values 128, 130.
(BRI: capturing images at the same wavelength generally leads to the same number of data points, as each pixel will represent the intensity of light at that specific wavelength. However, the number of data bits per pixel (radiometric resolution) can vary.. Using the value variations from 128 to 130, the range is 2/128= 1.5625 % (within 5 %))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and Blair.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
Blair teaches number of the extracted images from the authentic consumer goods is within 5% or less of a number the extracted images from the non- authentic consumer goods.
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and Blair that can help reduce the outlying data (Blair [0029])
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over
Tak Wai Lau et. al (hereinafter Lau) US 2020/0410510 A1,
in view of Samira Pouyanfar et.al (hereinafter Pouyan), Dynamic Sampling in Convolutional Neural Networks for Imbalanced Data Classification, 2018 IEEE Conference on Multimedia Information Processing and Retrieval.
in view of Madhusudan Krishnamoorthy et.al (hereinafter Krish) US 2020/0218825 A1.
further in view of Iain McDonald et.al (hereinafter McDonald) US 2019/0213462 A1.
In regards to claim 24: (Previously Presented)
Lau, Pouyan and Krish do not explicitly disclose:
- wherein the extracted images of the authentic product have a shape of a first polygon and wherein the extracted images of the authentic product augmented with the geometric distortion have a shape of a second polygon that is different from the first polygon.
However, McDonald discloses:
- wherein the extracted images of the authentic product have a shape of a first polygon and wherein the extracted images of the authentic product augmented with the geometric distortion have a shape of a second polygon that is different from the first polygon
[0005]:
traditional tags, the secure tags each contain a unique, discreet key, have dynamic and flexible areas of storage, are integrated with digital ledgers, and contain numerous other advantages. For example, the secure tags are not limited to one shape or forms—they can exist in multiple design states to fit the need of the customer.
[0155]:
During tag detection step 801, client device 110 can be configured to determine an orientation of a tag feature and rotate the image based on the determined orientation of the tag feature. The rotation can further be based on a target parameter value retrieved from the public portion of the stylesheet. The tag feature can a center logo of the secure tag. Client device 110 can be configured to identify a center of the tag using the template match system described above. Client device 110 can be configured to then determine an outer ovoid line encompassing the entire secure tag. Client device 110 can be configured to then determine a center of the secure tag. After determining the center and the ovoid line, client device 110 can be configured to construct multiple right triangles on the secure tag image. The right triangles can be placed such that the center of each right triangle overlaps the center of the secure tag, while the two vertices bounding the hypotenuse intersect the outer ovoid rim. The triangle(s) with the least and/or greatest hypotenuse can be used, in conjunction with orientation information in the public portion of the stylesheet, to correct the orientation of the tag.
[0026]:
the potential secure tag can include detecting image gaps by determining tag feature options for potential image gaps and comparing the tag feature option values to target parameter values. The target parameter values can include an inner or outer tag rim thickness and diameter ratio or an inner or outer tag rim thickness and tag rim break width ratio. In other embodiments still, generating a normalized image of the potential secure tag can include determining an orientation of a tag feature and rotating the image based on the determined orientation of the tag feature and a target parameter value retrieved from the stylesheet. The tag feature can be a center logo and the target parameter value can include a center logo orientation.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Lau, Pouyan, Krish and McDonald.
Lau teaches using a neural network model to classify the product authenticity after capturing the image of the product that contains a steganographic feature and outputting the result of the classification, providing data augmentation with geometric distortion.
Pouyan teaches using at least three different camera types for capturing the image.
Krish teaches balanced training set across the camera types.
McDonald teaches shapes of a polygon.
One of ordinary skill would have motivation to combine Lau, Pouyan, Krish and McDonald that can reduce the authentication issues across the supply chain (McDonald [0006])
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to TIRUMALE KRISHNASWAMY RAMESH whose telephone number is (571)272-4605. The examiner can normally be reached by phone.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached on phone (571-272-3768). The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be
obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format.
For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIRUMALE K RAMESH/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights