Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This Office Action is responsive to Applicants' Amendment filed on 11/18/2025, in which claims 1, 4, 14, and 20 are amended. Claims 2, 3, 15, 16, 18, 19, 22 and 23 are newly cancelled. No claims are newly added.
Claims 1, 4-6, 13, 14, 17, 20, 21, and 24 are currently pending.
Response to Arguments
With regards to the rejections of claims 1-4, 6, 13, 14, 17-20, and 24 under 35 U.S.C. 101 as directed towards abstract ideas, Applicant’s arguments that independent claims 1 and 14 as amended contain limitations previously found to be 101-eligible are found persuasive, and the rejections are withdrawn.
With regards to the rejections of claims 1, 2, 4, and 6 under 35 U.S.C. 102(a)(1) as anticipated by Jun et al. (Chinese Patent Application Publication No. 109635634), Applicant’s arguments that the claims as amended overcome the rejections are found persuasive, however the arguments are moot in light of a new grounds of rejection, necessitated by Applicant’s amendments to the claims, as presented below.
With regards to the rejections of claims 3, 13, 14, 17-20, and 24 under 35 U.S.C. 103 as unpatentable over Jun, in view of Kim et al. (U.S. Patent Application Publication No. 2020/0302171), and the rejections of claims 5, 15, and 16 under 35 U.S.C. 103 as unpatentable over Jun, in view of Taylor and Nitschke “Improving Deep Learning using Generic Data Augmentation”, Applicant’s arguments that the claims as amended overcome the rejections are found persuasive, however the arguments are moot in light of a new grounds of rejection, necessitated by Applicant’s amendments to the claims, as presented below.
Prior Art
The following references are used for prior art claim rejections:
Jun et al. (Chinese Patent Application Publication No. 109635634)
Taylor and Nitschke “Improving Deep Learning using Generic Data Augmentation”
Dwibedi et al. “Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-6, 13, 14, 17, 20, 21, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Jun et al. (Chinese Patent Application Publication No. 109635634), hereinafter Jun, in view of Taylor and Nitschke “Improving Deep Learning using Generic Data Augmentation”, hereinafter Taylor, further in view of Dwibedi et al. “Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection”, hereinafter Dwibedi.
Regarding claim 1,
Jun teaches A data augmentation method, comprising: ((Jun [0002]) “The present invention relates to the fields of video surveillance and data processing, and in particular to a pedestrian re-identification data enhancement method based on random linear interpolation”)
selecting at least two different sets of samples from an original data set, ((Jun [0047]) “Step 1: Use the monitoring system to capture photos of the same pedestrian from different cameras with non-overlapping viewing angles, then extract the pedestrian images from different pedestrian photos to form an image dataset of the pedestrian; use the image datasets of different pedestrians to construct a pedestrian re-identification dataset and divide it into a training set and a test set”, a training set and a test set from an image dataset are two different sets of samples)
each set of samples including input samples and output samples; ((Jun [0072]-[0073]) “we introduce two standard person re-identification datasets Market1501 and DukeMTMC-reID…Market1501 dataset: The Market1501 dataset is made up of data collected by six cameras in front of a supermarket at Tsinghua University…There are about 20 images of each pedestrian, and then the bounding box image data of 1501 pedestrians are labeled using the hand-crafted and deformable part model (DPM) methods”, images of each pedestrian are input samples, labels for each pedestrian are output samples)
generating at least one random number ((Jun [0052]) “Step 2.1, first generate the random linear interpolation intensity μ through Beta distribution Beta(α, β), where α = β is the parameter of Beta distribution”) greater than 0 and less than 1 (Jun Fig. 2 shows that the random number μ for all example images has a value between 0 and 1) …
PNG
media_image1.png
790
1062
media_image1.png
Greyscale
and generating at least one extended input data sample according to input samples in the at least two different sets of samples and the at least one random number, ((Jun [0051]) “set the proportion of samples that need to be enhanced in the training set, interpolate the sample to be enhanced with a random sample in the training set to generate a new sample”)
and generating at least one extended output data sample according to output samples in the at least two different sets of samples and the at least one random number, ((Jun [0051]) “interpolate the sample to be enhanced with a random sample in the training set to generate a new sample, and then relabel the sample to be enhanced”, a label is an output sample)
each extended input data sample corresponding to a respective extended output data sample; ((Jun [0056]) “In step 2.3, the interpolation strength μ in step 2.1 is used to relabel the samples to be enhanced in the batch training data to obtain the double labeling of the enhanced samples”, enhanced samples with new double labels correspond to each extended input data sample corresponding to respective extended output data samples)
Taylor teaches the following further limitations that Jun does not explicitly teach:
wherein before selecting the at least two different sets of samples from the original data set, ((Taylor Pg. 4) “Specifically 4-fold4 cross-validation was used which partitioned the data-set into 4 equal sized subsets, where 3 subsets were used for training and the other for validation purposes”) the data augmentation method further comprises: performing a second image processing on input samples in the original data set, the second image processing including changing…brightness of the images of the input samples ((Taylor Pg. 1) “Photometric transformations amend the color channels with the objective of making the CNN invariant to change in lighting and color. For example, color jittering and Fancy Principle Component Analysis (PCA)”, (Taylor Pg. 2) “this study’s objective is to evaluate a variety of popular geometric and photometric augmentation schemes on the coarse grained Caltech101 data-set”)
At the time of filing, one of ordinary skill in the art would have motivation to combine Jun and Taylor by taking the method for data augmentation taught by Jun, and combining it with the method for data augmentation including changing the lighting and color, including the brightness, of images taught by Taylor, as Taylor teaches that this technique (color jittering) increases the accuracy of models trained on datasets using it over a baseline dataset without it (see Taylor Pg. 5, Table 3). Such a combination would be obvious.
PNG
media_image2.png
238
487
media_image2.png
Greyscale
Dwibedi teaches the following further limitations that neither Jun, nor Taylor explicitly teach:
generating at least one random number…according to a uniform distribution ((Dwibedi Pgs. 5-6) “The objects are rotated at uniformly sampled random angles in between 30 to −30 degrees to account for camera/object rotation changes. Table 1 shows a gain of 3 AP points by adding this augmentation”, a random angle between 30 and -30 degrees is a random number)
the second image processing including changing directions and positions of different objects in images of the input samples, and a ratio…of the images of the input samples ((Dwibedi Pg. 7) “We generate a synthetic dataset with approximately 6000 images using all modes of data augmentation from Section 5. We sample scale, rotation, position and the background randomly”, Dwibedi Pg. 2, Fig. 2 shows that Dwibedi’s image processing includes changes to objects within input samples, rotation of an object corresponds to its direction, and scale of the object within the background corresponds to a ratio of the object’s size to the background’s)
PNG
media_image3.png
531
1382
media_image3.png
Greyscale
At the time of filing, one of ordinary skill in the art would have motivation to combine Jun, Taylor, and Dwibedi by taking the method for data augmentation including changes to image brightness jointly taught by Jun and Taylor, and combining it with the method for synthetic and augmented data generation including changes to object orientations, positions, and ratios relative to image background, taught by Dwibedi, as Dwibedi teaches (Dwibedi Pg. 8) “We presented a simple technique to synthesize annotated training images for instance detection. Our key insights were to leverage randomization for blending objects into scenes and to ensure a diverse coverage of instance viewpoints and scales” and (Dwibedi Pg. 4) “Our results show that our approach is competitive with the manual curation process, while requiring little time and no human annotation”. Such a combination would be obvious.
Regarding claim 4,
Jun, Taylor, and Dwibedi jointly teach The data augmentation method according to claim 1,
Jun further teaches:
wherein generating the at least one extended input data sample according to the input samples in the at least two different sets of samples and the at least one random number, and generating the at least one extended output data sample according to the output samples in the at least two different sets of samples and the at least one random number, includes: obtaining an extended input data sample through calculation according to x = α · x1 + (1 - α) · x2; ((Jun [0055]) “xa represents the features of the samples to be enhanced”, Jun Equation 1 on Pg. 5, paragraph [0015] of the original patent document shows that the features for a new input data sample are calculated in this way)
PNG
media_image4.png
46
740
media_image4.png
Greyscale
and obtaining an extended output data sample corresponding to the extended input data sample through calculation according to y = α · y1 + (1 - α) · y2; ((Jun [0058]) “ya represents the label of the sample a to be enhanced”, Jun Equation 2 on Pg. 5, paragraph [0018] of the original patent document shows that the features for a new output data sample are calculated in this way)
PNG
media_image5.png
33
702
media_image5.png
Greyscale
wherein α is a random number, ((Jun [0055]) “μ represents the interpolation strength of random linear interpolation”)
x and y are respectively the extended input data sample and the extended output data sample corresponding to the extended input data sample, ((Jun [0055]) “Among them, x̄a represents the features of new samples generated by batch training data”, (Jun [0058]) “Among them, ȳa represents a mixture of the labels of the sample a to be enhanced and any sample t in the batch training data, and has the label information of both samples”)
x1 and y1 are respectively an input sample and an output sample of a set of samples, ((Jun [0055]) “xa represents the features of the samples to be enhanced (i.e., data augmentation is required) in the batch training data”, (Jun [0058]) “ya represents the label of the sample a to be enhanced”)
and x2 and y2 are respectively an input sample and an output sample of another set of samples ((Jun [0055]) “xt represents the features of any sample in the batch training data”, (Jun [0058]) “and yt represents the label of the arbitrary sample t”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Jun, Taylor, and Dwibedi for the parent claim of claim 4, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 5,
Jun, Taylor, and Dwibedi jointly teach The data augmentation method according to claim 1,
Taylor further teaches:
wherein before selecting the at least two different sets of samples from the original data set, ((Taylor Pg. 4) “Specifically 4-fold4 cross-validation was used which partitioned the data-set into 4 equal sized subsets, where 3 subsets were used for training and the other for validation purposes”) the data augmentation method further comprises: performing a first image processing on input samples in the original data set, the first image processing including at least one of inverting, translating or rotating images of the input samples ((Taylor Pg. 1) “Geometric transformations alter the geometry of the image with the aim of making the CNN invariant to change in position and orientation. Example transformations include flipping, cropping, scaling and rotating”, (Taylor Pg. 2) “this study’s objective is to evaluate a variety of popular geometric and photometric augmentation schemes on the coarse grained Caltech101 data-set”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Jun, Taylor, and Dwibedi for the parent claim of claim 5, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 6,
Jun teaches A method of training a supervised learning system, comprising: ((Jun [0024]) “Put the samples of the test set into the model obtained in step 3 for training”)
augmenting a data set for training the supervised learning system based on the data augmentation method according to claim 1; ((Jun [0060]) “In step 3, the generated new samples are mixed with the samples in the training set as the input layer”)
and training the supervised learning system using the data set ((Jun [0061]) “In this embodiment, the new samples generated in step 2 and the original samples in the training set are put into the convolutional neural network for training”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Jun, Taylor, and Dwibedi for the parent claim of claim 6, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 13,
Claim 13 recites a non-transitory computer-readable medium storing computer program instructions for performing the function of the method of claim 1. Specifically, claim 13 recites: A non-transitory computer-readable storage medium having stored computer program instructions thereon, wherein the computer program instructions, when run on a processor, cause the processor to perform the data augmentation method according to claim 1. Dwibedi teaches (Dwibedi Pg. 7) “We use this synthetic data for all our experiments. The code used for generating scenes is available at: [URL]”. At the time of filing, one of ordinary skill in the art would have motivation to take the instructions for a computer to perform instructions jointly taught by Jun, Taylor, and Dwibedi and implement them on a medium comprising memory with those instructions encoded, as it is well-known within the art to encode instructions upon computer-readable media for distribution.
Regarding claims 14, 20, and 21,
Claims 14, 20, and 21 recite a computer device including a memory for storing results for performing the function of the method of claims 1, 4, and 5 respectively. Specifically, claim 14 recites: A computer device, comprising: a memory configured to store at least one of an initial result, an intermediate result, or a final result; and at least one processor configured to perform: [the method of claim 1]. Taylor teaches (Taylor Pg. 5) “All implementation was completed in Java 8 using DL4j with the experiments being conducted on a NVIDIA Tesla K80 GPU using CUDA”, with a GPU being a computer device comprising memory for storage and at least one processor. All other limitations in claims 14, 20, and 21 are substantially the same as those in claims 1, 4, and 5 respectively, therefore the same rationale for rejection applies.
Regarding claim 17,
Claim 17 recites a non-transitory computer-readable medium storing computer program instructions for performing the function of the method of claim 6. Specifically, claim 17 recites: A non-transitory computer-readable storage medium having stored computer program instructions thereon, wherein the computer program instructions, when run on a processor, cause the processor to perform the method of training the supervised learning system according to claim 6. Dwibedi teaches (Dwibedi Pg. 7) “We use this synthetic data for all our experiments. The code used for generating scenes is available at: [URL]”. At the time of filing, one of ordinary skill in the art would have motivation to take the instructions for a computer to perform instructions jointly taught by Jun, Taylor, and Dwibedi and implement them on a medium comprising memory with those instructions encoded, as it is well-known within the art to encode instructions upon computer-readable media for distribution.
Regarding claim 24,
Claim 24 recites a computer device including a memory for storing results for performing the function of the method of claim 6. Specifically, claim 24 recites: A computer device, comprising: a memory configured to store at least one of an initial result, an intermediate result, or a final result; and at least one processor configured to perform: [the method of claim 6]. Taylor teaches (Taylor Pg. 5) “All implementation was completed in Java 8 using DL4j with the experiments being conducted on a NVIDIA Tesla K80 GPU using CUDA”, with a GPU being a computer device comprising memory for storage and at least one processor. All other limitations in claim 24 are substantially the same as those in claim 6, therefore the same rationale for rejection applies.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Inoue “Data Augmentation by Pairing Samples for Images Classification” teaches a method of data augmentation by synthesizing new images from pairs of existing images.
Kim et al. (U.S. Patent Application Publication No. 2020/0302171) teaches a method of training a neural network using a set of images which are homographically augmented.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTOR A NAULT whose telephone number is (703) 756-5745. The examiner can normally be reached M - F, 12 - 8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.A.N./Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights